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ABSTRACT 

An experimental study tested a model for enhancing mathematics instruction in five high 
school career and technical education (CTE) programs (agriculture, auto technology, business/ 
marketing, health, and information technology). The model consisted of a pedagogy and intense 
teacher professional development. Volunteer CTE teachers were randomly assigned to an 
experimental ( n = 57) or control (n = 74) group. The experimental teachers worked with math 
teachers in communities of practice to develop CTE instructional activities that integrated more 
mathematics into the occupational curriculum. After 1 year of the math-enhanced CTE lessons 
averaging 10% of class time, students in the experimental classrooms performed significantly 
better on 2 tests of math ability-the TerraNova and ACCUPLACER®-without any negative 
impact on measures of occupational/technical knowledge. 
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EXECUTIVE SUMMARY 

Many high school students, particularly those enrolled in career and technical education 
(CTE) courses, do not have the math skills necessary for today’s jobs or college entrance 
requirements. Math is found in all areas of CTE, but is largely implicit to both teachers and 
students. This report describes a group randomized trial (GRT) research study designed to test a 
model for enhancing mathematics instruction in high school CTE courses emphasizing the 
mathematics that is already embedded in the CTE curriculum. The aim was to help CTE teachers 
make mathematics more explicit in a meaningful context and then help reinforce students’ 
mathematics understanding both in and out of that context. 

We hypothesized that conceptual mathematics learning and transferability of skills could be 
enhanced by using a contextual approach, and that testing students on both traditional (abstract) 
and applied math problems would show whether this was accomplished. The creation of explicit 
connections between situations is critical if students are to transfer their knowledge and skills 
outside the classroom, whether it is to another context or to a testing situation. We call this 
approach contextual. Unlike other models that are context-based, the mathematics in our 
contextual model arose from the CTE curriculum, rather than being forced into it. 

The Math-in-CTE model, developed by the National Research Center for Career and 
Technical Education (NRCCTE), consists of both teacher professional development and a seven- 
element pedagogy. The process involved professional development in which groups of CTE 
teachers from each of five occupational areas (agriculture, auto technology, business/marketing, 
health, and information technology) worked with math teachers in examining the CTE 
curriculum and identifying the embedded mathematical concepts. The CTE and math teachers 
worked together in teams to develop CTE instructional activities that would enhance the teaching 
of mathematics that already existed (but was previously not emphasized) in the CTE curriculum. 
The seven-element pedagogy was designed to move CTE students gradually from a contextual 
understanding of mathematics to a more abstract or traditional understanding such as that often 
reflected in standardized tests. No commercially available curricula were suited to test this 
particular pedagogical model; therefore, teachers needed to develop their own lessons. We 
believe this early investment on the part of teachers was a critical component to the success of 
the model. 

Volunteer teachers were recruited and randomly assigned to an experimental or control 
group. Because random assignment was conducted at the teachers’ classroom level, rather than at 
the individual student level, the unit of analysis in this research study was the classroom. This 
design is called a group randomized trial (Murray, 1998) and calls for analysis of student math 
scores aggregated to the classroom level. A total of 131 CTE teachers took part in this study: 57 
teachers in the experimental group and 74 in the control group. Almost 3,000 students in those 
teachers’ classrooms also participated. 

During the 2004-2005 school year, the experimental CTE teachers taught the math-enhanced 
lessons they had developed in their professional development workshops. Teachers in the control 
condition were asked to teach their regular CTE curricula with no changes. Participants in both 
conditions were paid. Although random assignment should theoretically yield equal groups, 
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pretesting of students was done in fall 2004 to ensure equality of classroom averages at the start 
of the school year. Pretest scores were then used as a covariate in the analyses, as is typically 
done despite the assumption of group equality due to random assignment (Fraenkel & Wallen, 
2003). Three different types of posttests were administered at the end of the school year (spring 
2005) after all of the enhanced lessons had been taught: TerraNova (a global, standardized test of 
math ability), ACCUPLACER (a college placement exam), and WorkKeys® (a test of applied 
mathematics ability). In addition, students in each of the five occupational areas took a posttest 
that assessed their occupational knowledge and skills in that area; these tests were administered 
to determine whether or not the instruction time used for enhancing math was detrimental to the 
learning of the CTE content. 

Both quantitative and qualitative data were collected and analyzed to assess fidelity of the 
treatment and to gain understanding about experimental teachers’ experiences during 
implementation of the model. Teacher surveys and focus groups were conducted. CTE-math 
teacher-teams were asked to meet before each lesson and submit reports after the lesson was 
taught. Additionally, each teacher was observed once during the semester by a member of the 
research team, and instructional artifacts were collected from each classroom. 

After 1 year of exposure to the math-enhanced lessons, the students in the experimental 
classrooms performed significantly better on the TerraNova and ACCUPLACER tests of math 
ability. They also performed better on WorkKeys, though the difference was not significant. 
Furthermore, there were no differences in measures of occupational or technical 
knowledge — meaning that CTE students’ math skills increased without detracting from the 
content skills learned in their CTE courses. 

The results presented in this report were achieved without the need for exemplary school- 
based leadership or cultural change within the school, as opposed to what is commonly 
concluded from other school reform literature. Instead, the improved math performance of the 
experimental students was produced by assembling teams of teachers in a single occupational 
area and providing them with a process and a pedagogy through which they could successfully 
enhance the math in their own curricula. Essential to the model was the ongoing teamwork 
between CTE instructors and their math partners in an authentic community of practice. 



xii 



National Research Center for Career and Technical Education 




Building Academic Skills In Context: Testing the Value of Enhanced Math Learning in CTE 



CHAPTER 1: INTRODUCTION 

This study tested whether a new pedagogy combined with professional development 
designed to enhance mathematics instruction in different occupational contexts would influence 
student performance on standardized tests of mathematics. The courses involved in the study 
were formerly called vocational education and are now referred to as career and technical 
education (CTE). 1 Numerous high school students, including many who are enrolled in CTE 
courses in specific labor market preparation (SLMP) areas, do not have the math skills necessary 
for today’s high skill workplace or college entrance requirements. The aim of this study was to 
help CTE teachers make the mathematics already embedded in the occupational curriculum more 
explicit as a necessary tool for solving workplace problems and then help reinforce students’ 
mathematics understanding both in and out of that context. 

Background 

The forces of technology, demographics, and global economic competition are demanding 
higher levels of problem-solving skills among workers. Unlike jobs a half-century ago, most 
positions today that pay family- supporting wages and offer opportunities for advancement 
demand strong academic and technical skills, technological proficiency, and some education 
beyond high school. A recent report from ACT (2004) found that most students are not being 
prepared to meet these demands: 

ACT research shows that far too few members of the graduating class of 2004 are 
ready for college-level work in English, math, or science — or for the workplace, 
where the same skills are now being expected of those who do not attend college. 

This deficiency is evident among both males and females and among all racial 
and ethnic groups. And, at present, it does not look as though students already in 
the pipeline are likely to fare much better, (p. 1) 

In 2000, the National Council of Teachers of Mathematics (NCTM) issued a report that 
emphasized math as one of the new basic skills (Murnane & Levy, 1996) for industry. 
Mathematics is no longer a requirement only for prospective scientists and engineers. Instead, 
some degree of mathematical literacy is required of anyone entering a workplace or seeking 
advancement in a career (National Research Council, Mathematical Sciences Education Board, 
1995). Research by Levy and Murnane (2004) has shown that higher wages depend on the ability 
to think mathematically. 

The calls for reform are continuous, as are employers’ complaints about the difficulty of 
hiring young people who have the right skills: 

Seventy-eight percent of respondents believe public schools are failing to prepare 
students for the workplace, which represents little change from the 1991 and 1997 



1 In 1999, the American Vocational Association changed its name to the Association for Career and Technical 
Education. Most state and local agencies have adopted the new name. In the present report, the field will be referred 
to as career and technical education (CTE). 
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surveys, despite a decade of various education reform movements. Respondents 
said the biggest deficiency of public schools is not teaching basic academic and 
employability skills. (National Association of Manufacturers, 2001, p. 2) 

The results of the National Assessment of Educational Progress (NAEP) support this 
criticism. The most recent results indicate that 37% of 12th-grade students performed at a Below 
Basic level on the math portion of the test. An additional 45% performed at a Basic level, and 
only 18% were Proficient or above (Perie & Moran, 2005). What is more, there was very little 
improvement in the decade from 1990 to 2000. 

International comparisons of student performance also underscore the need to improve the 
math skills of United States students. The TIMSS (Third International Mathematics and Science 
Study) tests which were administered at the fourth and eighth grades in 43 countries in 2003 
found United States students in the middle of the distributions in both grades when compared to 
countries at similar levels of economic development (Gonzales et al., 2004). On the 2003 
TIMSS, United States’ 12th-grade students scored about 50 points below the international 
average in advanced math, including numbers & equations, calculus, and geometry. On the PISA 
(Programme for International Student Assessment) test in 2003, the math scores of 15-year-olds 
in the United States ranked 25th among 40 industrialized countries (Organisation for Economic 
Co-operation and Development, 2004). 

Analyses of other data reveal that only 30% of all students complete the minimum courses 
recommended for college entrance, and nearly one-half of postsecondary students require 
remedial coursework once they get to campus (Delci & Stern, 1999). Almost three fourths of 2- 
and 4-year postsecondary schools offer remedial math courses, with a 22% freshman enrollment 
rate (Levesque, 2003). However, remedial classes at the college level are not a long-term 
solution but rather a bandage. Furthermore, what is the condition for students who do not enroll 
in postsecondary education? United States’ high school juniors and seniors are clearly 
unprepared for the math they will need in all settings after they graduate. 

The obvious solution to the problem, requiring more mathematics courses in high school, 
may not be effective. The NAEP assessments of mathematics show a flat growth curve over the 
past three decades (the average score of 17-year-olds was not measurably different in 2004 than 
in 1973; Perie & Moran, 2005). During this time, school districts across the nation have 
increased academic coursework, mostly in math and science, by four Carnegie units (see 
Levesque, 2003), and the high school completion rate has been on a slow and steady decline 
(Swanson, 2004). A recent study of teaching practices across the countries involved in the 
TIMSS study found that United States teachers tend to focus more often on the execution of low- 
level math skills compared with higher-achieving countries that used different methods and 
emphasized conceptual understanding, procedural skill, and challenging content (Hiebert et al., 
2005). Together, these data suggest that doing more of the same is not an effective strategy for 
improving the math skills of high school students. 
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One Potential Solution 

Nearly all high school students take at least one CTE course during their high school 
experience (Silverberg, Warner, Fong, & Goodwin, 2004). There are numerous studies that 
suggest participating in CTE provides a number of economic and academic benefits to high 
school students (see Bishop & Mane, 2004; Hollenbeck, 2003; Plank, 2001; Plank, DeLuca, & 
Estacion, 2005; Silverberg, Warner, Fong, & Goodwin, 2004; Stone & Aliaga, in press; see also 
Alfeld et al., forthcoming, and Castellano, Stringfield, & Stone, forthcoming). Like other high 
school students, most CTE students will attempt college within the first 2 years following 
graduation (Silverberg et al., 2004) but will encounter obstacles, primarily due to poor math 
skills (Rosenbaum, 2002). CTE courses that teach skills needed for occupational pathways 
(specific labor market preparation, or SLMP) are typically offered during the last 2 years of high 
school, during which time students have likely stopped taking math and are preparing for 
transition to post high school work and education (Plank, 2001; Plank, DeLuca, & Estacion, 
2005). Levesque (2003) documented this, showing that 36% of CTE credits were earned in the 
students’ senior year, and 24% during the junior year. These enrollment patterns suggest an 
opportunity to help youth who are beginning an occupational pathway in an SLMP to increase 
their math skills by highlighting the math embedded within their CTE coursework during those 
years. 

There is another reason to focus on CTE: students who invest in these courses come 
disproportionately from groups that are at risk of not successfully completing high school 
(Levesque, 2003). More than 70% of youth in poor communities take three or more SLMP 
courses, as do nearly 70% of Black students and 60% of Hispanic students. Students with 
disabilities, limited English proficiency, and low academic achievers are also overrepresented in 
CTE. Thus it is no surprise that many national studies show that, as a group, CTE concentrators 
generally achieve lower scores than students in the academic curriculum on tests of cognitive 
ability (Oakes, 1985; Plank, 2001; Stone & Aliaga, 2003). 2 In short, CTE serves large numbers 
of youth who do not prosper in traditional academic environments and about whom the 
discussion of closing the achievement gap is a major focus. 

Overview of the Study 

In this study, we hypothesize that we can enhance students’ mathematics learning in 
occupational contexts (CTE courses) and thereby improve math achievement among CTE 
students without the loss of technical skill achievement. Indeed, the National Research Council 
and the National Academies of Science encourage the design of engaging curricula that apply to 
real-world situations (National Research Council, Commission on Behavioral and Social 
Sciences and Education [NRC, CBASSE], 2000). Within each SLMP area, mathematics can be 
taught in the context of that occupation (CORD, 1999). For example, horticulturists estimate the 
number of pots of various diameters that can fit in an area of a greenhouse, a problem that uses 



2 Some research (e.g.. Stone & Aliaga, in press; Silverberg et al., 2004) shows CTE students to be taking more math 
than they have in the past and more math than a comparable group of non-CTE, non-college prep students. They still 
lag behind college preparatory students in the number of algebra I, algebra II and geometry courses taken (Delci & 
Stern, 1999). 
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math skills to determine the area of the pot and the surface. Many such examples can be drawn 
from across CTE areas. 

Since CTE educators are not trained to teach math, however, explicit math content, such as 
algebraic formulas, rarely makes it onto the blackboard. It should. Under the 1998 re- 
authorization of the Carl D. Perkins Vocational and Applied Technology Education Act (Perkins 
III), CTE classes are responsible for increasing students’ academic performance. More 
specifically, Perkins III (1998) states that an indicator of CTE performance is “student attainment 
of challenging State established academic... proficiencies” (Sect 113 item 2Ai). As such, CTE 
programs, and therefore educators, are expected to improve students’ academic performance. 

The research described in this report was funded by the Office of Vocational and Adult 
Education (OVAE), whose vision for vocational education supports and extends the goals of the 
No Child Left Behind Act of 2001. The study targets 5 of the 18 SLMPs that represent the depth 
and breadth of CTE (auto technology, agriculture, health, IT, and business/marketing) and uses a 
group randomized trial design (Murray, 1998). 

As described above, many of the academic skills that are required for both workplace success 
and entry into higher education — skills like algebra and other mathematical concepts — are taught 
late in middle school and early in high school (Rosenbaum, 1992). Typically there is little 
follow-up or reinforcement of these important skills for students in their later high-school years. 
As a result, most CTE students are not exposed to the higher levels of math they will eventually 
need after graduation. How can the math skills of these students be enhanced during this critical 
juncture without detracting from the CTE skill-building they need for the workplace? To answer 
this question, we sought to test the basic hypothesis that high school students in a contextual, 
math-enhanced CTE curriculum will develop a better understanding of mathematical concepts 
than will those students who participate in the traditional CTE curriculum. Specifically, we 
asked: 

1. Does a math-enhanced CTE curriculum improve student math performance as measured 
by traditional and applied tests of math knowledge and skills? And, does an enhanced 
CTE curriculum decrease students’ likelihood of requiring postsecondary math 
remediation? 

2. Does enhancing a CTE curriculum with mathematics reduce students’ acquisition of 
SLMP technical skills or knowledge? 

We attempted to answer these questions by proposing and experimentally testing a model for 
enhancing math instruction in an occupational context. 

We posited that using a contextual, applied approach can enhance conceptual mathematics 
learning and transferability of skills. CTE courses inherently provide contexts for applied or 
experiential learning (Owens & Smith, 2000; C. R. Rogers, 1969). Applied learning is the 
delivery of content area curricula within a relevant, authentic, and presumably more motivating 
context. Mathematical concepts are embedded in almost every CTE program, but there is no 
evidence to suggest that this situated math transfers to the workplace, other educational settings, 
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or real-world problems. 3 As will be explained further in Chapter 2, the creation of explicit 
connections between situations is critical if students are to transfer their knowledge and skills 
outside the classroom, whether it is to another context or to an abstract testing situation. In short, 
the mathematics in the CTE curriculum is implicit, both to the teachers and to the students; 
therefore, we designed and tested a model to make it explicit. 

The Math-in-CTE model we created for this study began with the principle that the math 
content ought to emerge from the occupational content rather than from superimposing math into 
the curriculum of a particular SLMP course. The goal was for CTE teachers to identify math 
concepts inherent in their curriculum and to move students from specific occupational 
applications of this math to the broader mathematical principles that these applications involve. 
Our goals for the students were that they be able to: recognize how to solve practical problems 
by using mathematics in their occupational area; recognize math occurring in other contexts; and 
do so without diminishing the acquisition of technical knowledge in the course. Students were 
tested on both typical (abstract) and applied math problems, as well as on their technical skills, to 
ascertain whether this was accomplished. (The methods we used are described fully in Chapter 

3.) 



This endeavor required the creation of a process and a pedagogy through which the CTE 
teachers could learn to develop and teach the math-enhanced CTE lessons. A systematic review 
of curriculum materials available through commercial and nonprofit vendors found none that 
were consistent with our operational definition of contextual learning (discussed further in 
Chapter 2). 4 The model we created involved professional development in which CTE teachers 
partnered with math teachers to: build curriculum maps that intersected math concepts with CTE 
curricula; create scope and sequence charts to identify when math enhancement opportunities 
would occur; and develop sets of math-enhanced CTE lessons for implementation in the CTE 
classrooms based on a specific pedagogical framework using seven elements. Importantly, 
during this professional development process, we were careful to emphasize that CTE teachers 
were not math teachers but rather CTE teachers teaching math skills for use in context. 

Development of the Math-in-CTE model was predicated on the assumption that the students 
participating in this study had already been introduced to algebraic and other procedural 
knowledge via their mathematics coursework during junior high and their freshman and 
sophomore years of high school. The Math-in-CTE model provided a pedagogic framework that 



3 For a variety of reasons, moreover, U.S. high school students need to be able to apply their math knowledge to 
paper-and-pencil tests, which are critical for school funding at the state level under the No Child Left Behind Act. 

While we found several examples of what we later identified as “context based” rather than contextual, we were 
unable to locate any standard curriculum (texts, lessons, approaches to professional development) that met our 
requirements. Two realizations came from this effort. First, even if an existing curriculum could have been located, 
we would have needed to provide professional development to prepare teachers to properly use the materials. We 
also came to understand that there is no national curriculum for any CTE area. That means that if curriculum 
materials had been located, adaptation would have been necessary in any event. In addition, as we discovered, some 
of the CTE teachers were unfamiliar with the math or methods of teaching it in context and the intensive 
professional development model we created was, in part, designed to address this. As a result, the research team 
reconceptualized the approach to be used and created a replicable pedagogic and professional development model. 
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makes explicit the mathematics concepts that naturally occur in occupationally specific CTE 
courses by gradually moving from the fully embedded example in CTE toward less 
contextualized and more abstract examples of the math concept. Through this contextualized 
approach, CTE teachers would refresh and reteach this knowledge through the math-enhanced 
lessons so that students would see the math as an essential component of the CTE content, a 
tool — like a saw, wrench, or thermometer — needed to successfully solve workplace problems. 

This approach is well supported in the literature. In CTE, students learn their trade in the 
context of actual work problems, and they perform best in areas in which their learning can be 
applied (Orr, Thompson & Thompson, 1999; Slaats, Lodewijks, & van der Sanden, 1999). 
According to David Kolb, less than 25% of students are abstract learners (CORD, 1999). For the 
remaining 75% of students, enhancing math in the CTE classroom can provide a valuable 
learning opportunity. The assumption was that if the teacher modeled a metacognitive approach 
to problem-solving (i.e., the process by which one consciously recalls how to solve similar 
problem situations), it was likely that students would think back to their lesson on the T-square 
in carpentry class when seeing the Pythagorean theorem in a paper-and-pencil test, and that they 
would remember how to solve the problem. 

This study addresses a gap in the literature on teaching secondary school mathematics. The 
1989 NCTM Principles and Standards for School Mathematics announced a departure from rote 
memorization in math learning and urged teachers to focus more on student engagement and 
realistic math problems. Teachers following the NRCCTE model made the math in their lesson 
explicit in order to promote a stronger linkage between what students learned in a particular 
project situation and the abstract concept behind it (NRC, CBASSE, 2000). Findings that 
demonstrate how such an enhanced CTE curriculum can build skill in key academic areas such 
as mathematics could aid in shaping future federal educational policy, especially in expanding 
opportunities for CTE participation in our secondary schools. In addition, such a finding would 
provide research-based evidence for the design of future programs that support the 
administration’s goal that every student will complete high school with the academic knowledge 
and skills they need to make a successful transition into postsecondary education or training 
without needing remediation at the college level. 

This study directly supports the President’s recently launched math and science initiative. 
Educators are seeking better ways to teach subjects such as mathematics, and it is our contention 
that academic classes alone are not fulfilling the mission for many high school students. In the 
following chapter, we further summarize the rationale behind the study and review the published 
research supporting our approach. In Chapter 3 we describe the research design and procedures 
for our experimental intervention. Chapter 4 describes how we measured/zV/e/zfy of treatment — 
the extent to which the interventions were implemented across the five replications. Chapter 5 
presents quantitative evidence of the effectiveness of the implementation as measured by three 
different standardized tests of mathematics achievement and a test of technical skills and 
knowledge, and Chapter 6 describes the results of the qualitative data analysis. In Chapter 7 we 
present our conclusions and examine the implications of the findings. 
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CHAPTER 2: TEACHING MATH-IN-CTE 

The National Council of Teachers of Mathematics (NCTM, 2000) outlines mathematics 
education goals commonly used throughout the country in their document, Principles and 
Standards. This national collaboration of teachers of mathematics states a goal of primary 
education: that all students will learn a rigorous core of mathematics that prepares them for work 
or postsecondary education. However, the NCTM makes it clear that wanting all students to 
learn math does not mean that all students can or should learn math in the same way. One 
possible solution is to develop other approaches that build on the traditional Algebra I course so 
as not to sacrifice the rigor of the current high school math program but make Algebra (or other 
math constructs) more accessible to the students who may be passing their required Algebra 
classes but failing to truly master important mathematical concepts. 

Research has shown that disengagement or lack of interest is a factor in low student 
achievement (NCTM, 2000). Students may disengage from math because of difficulty with the 
subject, lack of support, or simply boredom. Students may disengage while still attending class. 
Many of these students believe that the math that they learn in school is not relevant to life after 
high school (NCTM, 2000). Varying the curricular opportunities for high school youth may be a 
way to facilitate acquisition or mastery of Algebra or other math concepts. 

Building Math Skills in High School 

There is another reason to try teaching math in a different way. Students who lack a 
fundamental understanding of algebra and possess only a formulaic understanding of the course 
will struggle with applying the formulas in a testing environment, thereby differentially affecting 
this group’s graduation and college entrance rates. With the prevalence of high stakes and other 
standardized tests in the country today, teachers often feel pressured to teach only those topics 
likely to appear on the tests. This may lead to a superficial level of math instruction, especially 
when coupled with a lack of preparation on the part of the teacher (Viadero, 2005). According to 
Telese (2000), this is especially common in inner city and impoverished schools: schools that 
disproportionately serve minority students. 

Possible Solutions 

Increased Rigor of Mathematics Requirements 

Most school districts have responded by making Algebra I compulsory. For example, as part 
of its Equity 2000 initiative, Milwaukee Public Schools (MPS) required all of its ninth graders to 
take Algebra I with the exception of those who have already completed the class or whose 
Individual Education Plans recommend an exemption (Ham & Walker, 1999). As a result, 
algebra enrollment went from 31% to 99% in 1997. During the same period, algebra-passing 
rates for ninth graders went from 25% to 55%. Still, an average of 47% of ninth graders failed 
algebra I over the course of the period in which MPS implemented its Equity 2000 initiative. 

This indicates that dramatically more students are capable of succeeding in Algebra I than would 
enroll in the class on their own, however, the high failure rate may be indicative of larger issues 
surrounding student preparation, class instruction and curriculum design (Ham & Walker, 1999). 
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These and similar findings support our earlier contention that doing more of the same has not 
proven to be a viable solution to improving adolescents’ math skills. 

Curriculum Integration 

Integration of academics into the CTE curriculum is a major policy objective of the Carl 
Perkins Vocational Education Act (1985, 1990, 1998) (Hoachlander, 1999). Indeed, CTE courses 
hold promise as another venue in which to reinforce students’ math understanding and skill. 
However, despite the federal mandate, there is still no agreement on what curriculum integration 
should look like. There have been several efforts in CTE circles to define curriculum integration 
(Cl) models (Grubb, Davis, Lum, Plihal & Morgaine, 1991; Hoachlander, 1999). However, as 
Johnson, Charner, and White (2003) observed, much of the available literature is descriptive, and 
quantifiable outcome data are scarce. 

All Cl models attempt to move away from the traditional model of instruction, in which 
subjects are taught by themselves, completely isolated from any context. Traditional 
mathematics, for instance, is seen as abstract, disconnected from any real application (Brown, 
Collins, & Duguid, 1989). In the case of algebra, the equations are presented as things to be 
solved or symbols to be moved around or graphs to be drawn without any discussion of the real- 
life applications of the math (Kieran, 1990). Some math educators believe that students have a 
lot of trouble learning algebra in a decontextualized way (Boaler, 1998; Kieran, 1992). For many 
students, it becomes too abstract too quickly and does not make any sense. This issue is 
particularly acute with low achievers (Woodward & Montague, 2002). Perhaps more than other 
students, low achievers need an authentic lesson as a way to make sense of the abstract 
mathematics. 

The goal of Cl is to demonstrate to students the connection between academic subjects. 
However, not all Cl models are the same. Some use a coordinated approach, such as in 
specialized career academies where all teachers coordinate their subjects around specific themes 
(e.g. health). This method corresponds to the definition offered by the Association for 
Supervision and Curriculum Development (ASCD), which is as follows: 

Integration is a philosophy of teaching in which content is drawn from several 
subject areas to focus on a particular topic or theme. Rather than studying math or 
social studies in isolation, for example, a class might study a unit called The Sea, 
using math to calculate pressure at certain depths and social studies to understand 
why coastal and inland populations have different livelihoods (ASCD, 2003). 

Still other integrated learning models are context-based and try to fit mathematics into CTE 
by changing the content of word problems. In these models, CTE is the context for delivery of 
traditional academics. Such approaches are also called related math or applied math. Attempts to 
contextualize the symbols in word problems may not be an adequate or effective fix for all 
learners, however, especially “students with mild disabilities and at-risk students, who have few 
resources to guide their problem-solving performance” (Montague, 1992, cited in Jitendra & 

Xin, 1997, p. 435). 
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We refer to the NRCCTE Math-in-CTE model as contextual (as opposed to context-based) in 
that math learning occurs within a real-world CTE context. Berns and Erickson (2001) define 
contextual learning as learning that involves students connecting the content with the context in 
which that content could be used. They emphasize this connection to bring meaning to learning. 
In the same light, Karweit (1993) defines contextual learning as learning that is designed to 
support students in activities and problem solving in ways that reflects the real-world nature of 
such tasks. The U.S. Department of Education’s Office of Vocational and Adult Education 
defined contextual learning “as learning that motivates students to make connections between 
knowledge and its applications to their lives” as family members, citizens, and workers (Bems & 
Erickson, 2001). 

The contextual mathematics approach requires that students become more actively engaged 
as learners and that educators change the way in which they deliver content in order to produce 
enhanced thinking about and use of mathematics concepts. According to this perspective, 
educators play a major role in helping students find meaning in their education and make 
connections between what they are learning in the classroom and ways in which that knowledge 
can be applied in the real world. The use of authentic situations serves to "anchor" the symbolic 
and abstract math in situations that are familiar and real to students, which serves to help them 
make sense of the content (Brown, Collins, & Duguid, 1989; Cognition and Technology Group 
at Vanderbilt, 1990). The National Science Foundation (NSF) has funded several contextual 
mathematics projects at the high school level, one of which is Core-Plus. Core-Plus has been 
found to increase students’ conceptual understanding and problem solving in applied contexts; 
however students in Core Plus do not score any higher than students in the traditional math 
curriculum on a placement test used at a major university (Schoen & Hirsch, 2003). 

Contextual Learning and Situated Cognition 

One problem with contextual learning is that students may be unable to transfer the 
knowledge learned in one context or situation to another context or situation because it is so 
embedded ( situated ) in the original context where it was learned (Boaler, 1993; 1998; Lave, 
1988; Lave & Wenger, 1991). The problem with authentic, contextual learning is that 
“knowledge is. . . dependent upon and embedded in the context and activity in which it is 
acquired and used” (Karweit, 1993, p. 54) and may not be transferable to other contexts. Unless 
students are taught the abstract principle behind what they are learning in context and guided 
through other contextual examples to which it applies, it is unlikely that cognitive transfer will 
occur outside the classroom (see Fuchs et al., 2003). 

This is a critical problem because students will eventually need to use their math skills 
outside the classroom. In 2002, the National Alliance of Business examined the skills needed to 
obtain jobs that offer reasonable pay and opportunities for advancement and that are likely to 
experience growth. They worked with the Educational Testing Services and employers to draft a 
list of general workplace requirements in English and mathematics (cf. Achieve, 2004). 
Employers emphasized a need for students to have experience with application of these skills. 
This implies that students need to practice math skills in a variety of ways so that they become 
proficient in knowing when and how to apply them. The workplace of today is filled with 
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complex problems and rapidly changing environments and technologies where “the ability to 
cope with the nonroutine is perhaps the only knowledge worthy of instructional design in many 
cases” (Derry & Lesgold, 1996, p. 791). Indeed, expert performance is characterized by an 
ability to adapt ones’ skills to novel situations and actively solve problems (Ericsson & 
Charness, 1994). 



The Math-in-CTE Model 

The curriculum integration model that evolved in this study is consistent with the definition 
and characteristics provided by Johnson et al. (2003) in which they identified curriculum 
integration as a 

Relationship between academic and occupational or career-technical subject 
matter that goes beyond what would normally occur in the delivery of either the 
academic or occupational/career-technical subject matter alone (p. V). 

We believe that in order for students to make links between concepts, they need to go 
through a process beginning with an introduction to solving a real, relevant problem; practicing 
on several similar examples; and then applying the concept to a more abstract problem. Basic 
skills can be learned by rote, but the more flexible knowledge needed to become skilled requires 
deliberate practice (Ericsson, Krampe, & Tesch-Romer, 1993). One possible way to create 
deliberate practice is by asking students to solve a problem repeatedly in ways different than the 
ones they had previously solved. Anderson (1996) might describe the process of readdressing a 
problem as a tuning stage. Brownell (1956) has called this meaningful habituation (cited in 
Allen, 2003). These theorists argue for learning problem-solving in a “real world” context and 
practicing both similar and novel problems on a continuum from more contextualized to more 
abstract, which should pave the way for students to be able to transfer their skills to new 
situations and environments. 

Our basic assumption was that the mathematics taught in CTE courses should arise directly 
out of the occupational content. We wanted students to see the math as an essential component of 
the CTE course content, a tool needed to successfully perform the tasks of the occupation. 
Therefore, we considered it essential to develop a model that CTE teachers could use to improve 
instruction in the math concepts embedded in their occupational curricula so that students would 
be able to apply their understanding to that context as well as to other situations. 

Pedagogy and Process 



Pedagogy: The Seven Elements 

The NRCCTE Math-in-CTE model, which was the experimental intervention in this study, 
involved both a pedagogy and a process. The pedagogy built on theories of contextual learning 
and transfer and was created by the research team and educational consultants to guide the 
development and instruction of math-enhanced CTE lesson plans. This framework was labeled 
the “7 Elements of a Math-Enhanced Lesson” (see Figure 1). Using lesson plans developed with 
these seven elements, teachers introduced the CTE lesson and the math concepts in it and 
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assessed students’ understanding of that math. The teachers then presented math problems that 
were embedded in the CTE lesson. Critical to this approach is that the math concepts be 
addressed when they arise naturally from the curriculum rather than wait until it is convenient. 

When feedback from the students indicated that they understood the application, the teachers 
presented additional related contextual examples. The teachers then reviewed the same concepts 
as the students would encounter them in traditional forms in a math class or on a test. The final 
two steps required the students to demonstrate their understanding of the math concepts and 
procedures, both in and out of context, through projects or quizzes directly linked to the lesson 
and as part of the formal assessment for the overall instructional units in which the math- 
enhanced lessons were taught. Figure 2 depicts the components of the seven elements, and 
Figure 3 shows their application to a specific mathematical concept, the Pythagorean theorem. 

The seven elements in our pedagogical framework have many parallels with Gagne’s 
Conditions of Learning (1965) and Hunter’s ITIP (Instructional Theory Into Practice) (1982). 
What differentiates our framework from these, however, is its increased emphasis on moving 
from specific contextualized applications to general mathematical principles. Elements 3 and 4 
move the instruction from the original embedded CTE problem to additional contextual 
applications. In Element 5 the instruction moves to traditional examples in order to reinforce and 
expand the math to include that which students are likely to encounter in standardized tests. 

The creation of explicit connections between situations is critical if students are to transfer 
their knowledge and skills outside the classroom, whether it is to another context or to an 
abstract testing situation (Fuchs et al., 2003). Bay (2000) suggests, “Teaching via problem- 
solving is teaching mathematics content in a problem-solving environment. 

Learning in this approach involves learning through a concrete problem and eventually 
moving to abstraction” (Bay, 2000). Teachers following the NRC framework made the math in 
their lesson explicit in order to promote a stronger linkage between what students learned in a 
particular project situation and the abstract concept behind it (NRC, CBASSE, 2000). For 
example, as depicted in Figure 3, when using a T-square in an agricultural mechanics class, we 
encouraged the teacher to show the class the formula a 2 + if = c 2 , i.e., the Pythagorean theorem 
(see Appendix A for more detail and other examples). 
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1. Introduce the CTE lesson. 

• Explain the CTE lesson. 

• Identify, discuss, point out, or pull out the math embedded in the CTE lesson. 

2. Assess students’ math awareness as it relates to the CTE lesson. 

• As you assess, introduce math vocabulary through the math example embedded in the 
CTE. 

• Employ a variety of methods and techniques for assessing awareness of all students, e.g., 
questioning, worksheets, group learning activities, etc. 

3. Work through the math example embedded in the CTE lesson. 

• Work through the steps/processes of the embedded math example. 

• Bridge the CTE and math language. The transition from CTE to math vocabulary should 
be gradual throughout the lesson, being sure never to abandon completely either set of 
vocabulary once it is introduced. 

4. Work through related, contextual math-in-CTE examples. Using the same math concept 
embedded in the CTE lesson: 

• Work through similar problems/examples in the same occupational context. 

• Use examples with varying levels of difficulty; order examples from basic to advanced. 

• Continue to bridge CTE and math vocabulary. 

• Check for understanding. 

5. Work through traditional math examples. 

Using the same math concept as in the embedded and related, contextual examples : 

• Work through traditional math examples as they may appear on tests. 

• Move from basic to advanced examples. 

• Continue to bridge CTE and math vocabulary. 

• Check for understanding. 

6. Students demonstrate their understanding. 

• Provide students opportunities for demonstrating their understanding of the math 
concepts embedded in the CTE lesson. 

• Conclude the math examples back to the CTE content; conclude the lesson on the topic of 
CTE. 

7. Formal assessment. 

• Incorporate math questions into formal assessments at the end of the CTE unit/course. 



Figure 1. The Seven Elements: Components of a Math-Enhanced Lesson 
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Figure 2. The National Research Center model: The “7 Elements” of a math-enhanced lesson. 
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Figure 3. Sample building trades math-enhanced lesson: Using the Pythagorean theorem. 
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Process: Professional Development 

The development of the pedagogical framework (the seven elements) was only one aspect of 
the NRC model. The experimental intervention also required the creation of a process through 
which the CTE teachers could learn to develop and teach the math-enhanced lessons. The 
process included partnering CTE teachers with math teachers, building curriculum maps that 
intersected math concepts with CTE curricula, providing professional development for the 
teacher-teams, and implementing the math-enhanced lessons. Each of these aspects of the 
process is described in more detail in Chapter 3. As will be explained in the final chapter, we 
found that if either one of the components (process or pedagogy) are missing from the model, 
student math skills will not be improved. 

The Math-in-CTE model builds upon the work of Johnson et al. (2003), whose investigation 
of curriculum integration for NRCCTE provided a useful definition and several salient findings 
that guided our approach. These included: 

• Curriculum integration works better in a single industry or theme (operationalized in this 
study as an SLMP) 

• Curriculum integration works better when implemented in the same site (which we 
operationalized with our CTE-math teacher-teams) 

• Curriculum integration requires leadership and resources (which were provided in this 
study by NRCCTE staff) 

Our basic requirement was that the mathematics taught in CTE courses should arise directly 
out of occupational content rather than forced into it. As mentioned earlier, it is this fundamental 
principle that differentiates this approach from others. Also critical to this approach is the 
recognition that CTE teachers are not mathematics instructors and need assistance in identifying 
the math in their curricula and developing lessons to teach it; this is why we paired them with 
math teachers. The following chapter describes our methodology in detail. 
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CHAPTER 3: CONDUCTING THE STUDY 

This study began as a single semester (spring 2004) test of the experimental intervention (the 
Math-in-CTE model) in six different occupational areas (SLMPs). The results were promising 
enough that the funding agency, the Office of Vocational and Adult Education (OVAE) of the 
U.S. Department of Education, consequently authorized a full-year study of the same 
intervention with minor modifications. This chapter describes the recruiting and procedural 
efforts from both the semester-long study (which we now call the pilot - see Stone, Alfeld, 
Pearson, Lewis, & Jensen, 2005) and the full-year study (fall 2004-spring 2005). 

Before beginning the pilot study, we formed an advisory committee of methodologists and 
the Center’s Project Officer to ensure that the study would contribute to understanding strategies 
for improving math skills in high school students. This group of experts met twice before the 
start of the pilot project to discuss details of the theory, methods, design, and implementation of 
the research. We also formed a team of researchers and teachers across multiple SLMP 
replication sites that included CTE teacher educators, math experts, educational psychologists, 
curriculum integration specialists and administrators, and both current and former secondary 
CTE and math teachers. Many of these individuals also then served as site facilitators. This team 
met via conference call once per month, as well as multiple times in person at the NRCCTE 
headquarters during the duration of both the pilot and the full-year study. 

Experimental Design 

Several large design issues were discussed by the advisory committee prior to the start of the 
pilot. The following describes the decisions that were made. First, this study was designed as a 
field experiment with random assignment of teachers to the experimental and control conditions 
(group randomized trial, Murray, 1998). The logic of randomization is that all the unmeasured 
factors that may affect performance on the dependent variable will be randomly distributed 
between the two groups. When this method is used and a statistically significant difference is 
found between the groups, it can be attributed to the experimental intervention (Cook, 2002; U.S. 
Department of Education, 2003). The control, or counterfactual, condition permits the researcher 
to measure what would be expected if the students in the experimental classrooms had not 
received the intervention. Specific issues arising with a group randomized design will be 
discussed in the analysis section in Chapter 5. 

CTE teachers were assigned at random to the experimental or control group within each 
SLMP; thus the primary unit of analysis was the classroom (i.e., aggregate class performance). 
This design is often used in other research fields when individuals are nested within groups 
(Murray, 1998). As an extra precaution, as is often done in random assignment studies (Fraenkel 
& Wallen, 2003), we used a pretest of student math ability to test whether our random 
assignment of teachers yielded comparable classrooms. 

The strategy of using a teacher-level unit of analysis had three advantages. First, it avoided 
the well-documented problems of parental opposition to research activities in schools (for 
extended discussions of random assignment studies, see Cook, 2005 and Stem & Wing, 2004). 
With classroom level assignment, all students received or did not receive the treatment and could 
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only opt out of the testing regimen (which very few did). The second reason is more important. 
Because CTE classes are often “singletons” in their schools, there would have been no control 
class to which students could have been assigned. Even had there been an alternative section or 
class, our randomization process insured that the control classrooms were at other schools. This 
was done to limit the crossover effect described by Cook (2005). 5 Finally, recruiting at this level, 
rather than a district or school level, facilitated our ability to quickly launch the study. 6 

Simultaneous Replications 

A second design decision was to conduct simultaneous replications of the same essential 
study in multiple SLMPs (six in the pilot and five in the full-year study; see Figure 4). 
Replication means the repetition of treatments in an experiment and is necessary to provide both 
an estimate of experimental error and a more precise estimate of the main effect, since the 
standard deviation of the mean = o 2 / n, where o 2 represents the true experimental error and n the 
number of replications (Yang, 2002). Schafer (2001) advocates for routine replications in field 
research since researchers have more limited control over experiments than in the laboratory. He 
argues: 



When results are consistent across several studies, there is a stronger basis for 
observed relationship(s) than the support that is available within each study by 
itself, since results that have been replicated are considered more likely to 
generalize (continue to be observed). It is also possible to compare the studies 
with each other to identify constructs that interact with, or moderate, relation- 
ships. Although these advantages exist whether or not the research includes 
experimental control, the opportunity to replicate a basic study design in multiple 
field contexts is more likely to be available to the applied researcher and is a 
technique that can lead to stronger inferences in any setting. Thus, it is recom- 
mended that persons who conduct field research try to include replication as a 
fundamental feature in their studies (http://pareonline.net/getvn. asp?v=7&n=15). 

Finally, the approach of multiple simultaneous replications was taken to address one of the 
key criticisms of experimental research in education. Critics argue that random-assignment 
studies are not only rare, but that researchers who conduct them typically evaluate high-quality 
programs that serve only a few children, often at a single site, making it hard to generalize 
findings to large-scale programs or more diverse populations of children (Magnuson & 
Waldfogel, 2005). 



5 One consequence of this decision is that it limits our ability to engage in sub-group analyses using student-level 
characteristics (e.g., gender, race/ethnicity) at the classroom level. 

6 Most commentators on large, national, experimental studies suggest that recmiting schools and establishing an 
experiment takes 3 years (Cook, 2002; Whitehurst, 2003). Due to delayed funding approval for the initial pilot 
study, we were compelled to recruit teacher participants and establish experimental sites within six months. After 
recmiting teachers for each SLMP, we then obtained school/district permission for each participating teacher, rather 
than the other way around, which may have taken much longer. 
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Each of the occupational areas (SLMPs) in our study was studied within multiple schools in one 
geographic site. Therefore, “site” can be assumed to be synonymous with “SLMP” — within each 
SLMP, multiple schools were studied (i.e., SLMP = site = multiple schools). The five SLMPs in 
the full-year study represented the breadth of CTE programming. 7 We replicated the study in one 
program (business and marketing) that is essentially classroom based, one program (auto 
technology) that is heavily skill-oriented, two programs identified as high-tech and high growth 
(health and information technology), and one program historically associated with CTE 
(agriculture). To the extent that our findings are consistent across these SLMP replications, we 
believe that the conclusions we reach can be generalized to most occupationally specific CTE 
programs. 




Figure 4. The five simultaneous replications of the full-year experimental study. 



Method 



Sample Recruitment 

Based on advice from our advisory committee, we established an initial minimum pool of 40 
CTE teachers who would be randomly assigned to either the experimental or control condition in 
each SLMP. This sample size was deemed appropriate from two perspectives: statistical and 
practical. We assumed that our intervention would yield at least a 3-point difference in the 
number of correct responses on any of math tests between the experimental and control 
classrooms based on our pilot study results. Using a widely available power calculator and 
establishing .05 as the alpha level, we determined that a sample size of 40 (20 experimental and 
20 control) would be sufficient to detect significant differences between groups. 



7 The pilot study had six SLMPs, but for reasons of administrative difficulty, one was dropped for the full-year 
study. 
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The practical limitations on the sample size are a function of three factors. First was our 
ability to recruit a sample pool in a short period of time. As noted earlier, we had less than three 
months to recruit teachers. The second factor is that in many states, there are not that many 
teachers from which to recruit. For example, one state was very interested in testing the model in 
the machine-trades SLMP but there were fewer than 30 programs/teachers in the whole state. 
Finally, adding sample size would have significantly affected the project budget. Instead, we 
sought to increase the power of the analysis by including a covariate: math pretest scores (Cohen, 
1989; Murphy & Myors, 2003). 

A study of this magnitude required the cooperation of national organizations involved in 
CTE; many local-and state-level administrators; and university researchers. Initial recruitment 
occurred through four principal sources: the Association of Career and Technical Education, the 
National Council of Teachers of Mathematics, the National Tech Prep Leadership group, and 
SkillsUSA. A recruitment letter was sent to teachers on mailing lists obtained from these 
organizations announcing a “program” to enhance CTE courses with more rigorous and explicit 
mathematics lessons. Interested teachers were given an application or directed to log onto our 
Web site to download an application to participate, which they could then mail, email, or fax to 
the NRCCTE. This effort was supplemented by more focused recruitment of teachers within 
target states. 

The final sites were spread across the country: the business/marketing SLMP replication was 
in a western state; the IT and health SLMP replications were in Midwestern states; the auto 
technology SLMP replication was in several eastern states; and the agricultural power and 
technology SLMP replication was in a southern state. Details about the efforts made by the 
NRCCTE and each of the SLMP replication sites to recruit teachers into the study can be found 
in the Technical Notes. 

Teacher Response/Application 

CTE teachers who were interested in the study completed an application that required their 
signatures as well as those of school administrators to indicate that the teachers’ participation in 
the study would be supported. The application also called for the identification of a math teacher 
who would partner with the CTE teacher if he/she were selected for the experimental group. 8 If a 
CTE teacher applied without designating a partner, SLMP facilitators identified math teachers in 
their geographic area who could play this role. 

As an incentive to participate in the study, teachers in the initial experimental group (both 
CTE and math teachers) were originally offered a $1,500.00 stipend for the 2003-2004 year, 
plus all costs for travel, food, and lodging to attend the workshops. Because the original study 
(designed for one academic year) was truncated to spring semester only due to a delay in 
research funding, the teachers ended up receiving this amount for only a semester of 
participation. When the decision was made that a full-year experiment would be conducted the 
following year, they were offered an additional $2,500 to continue in the study. CTE teachers in 



8 We allowed CTE teachers to apply without a math teacher partner, but the most successful partnerships were the 
ones that were chosen by the teachers themselves. 
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the control group, who were asked to teach their curriculum as usual but allow their students to 
be pretested and posttested, also received a stipend ($500.00 for what is now referred to as the 
pilot and $750.00 for the full-year study). At the end of the 2004-2005 school year, the control 
teachers were offered an opportunity to receive the professional development on math 
enhancement of their CTE curriculum. 

Final Sample 

Teachers. We were able to recruit at least 40 teachers in five of the original six SLMPs, but 
in one (SLMP A) only 30 completed the full application process for the original spring 2004 
study. For the pilot study, we randomly assigned half of the CTE teachers in each SLMP to the 
experimental group, and the other half to the control group. There were at least 20 teachers in 
each condition in each SLMP except one. This design yielded 236 CTE teachers, 104 math 
teachers, and 3,950 students across 12 states. 

The nature of the study design meant that self- selection bias by teachers was inevitable. 

Given the concern for protection of human subjects, which requires voluntary informed consent, 
there is self- selection in virtually all research projects. The teachers who volunteered to 
participate likely shared more measurable and unmeasurable attributes than teachers who did not, 
which might include a higher level of comfort in teaching mathematics. We acknowledge this as 
a limitation to the generalization of our findings but not to the validity of the group comparisons, 
because the random assignment process distributed self-selection attributes to both groups 
equally. 

As mentioned earlier, our semester-long study in Spring 2004 became a pilot study when 
additional funding for continued research was made available to test the effect of an academic- 
year-length intervention during fall 2004-spring 2005. The pilot study served to provide valuable 
lessons about how to modify the model as well as how to provide better support to the teachers 
and monitor fidelity of treatment (see Technical Notes at the end of this report for changes from 
the pilot to the full-year study). The decision to continue the study into a full year was not made 
in time to recruit a new group of teachers for the 2004-2005 study, so we invited teachers who 
were already in the study in spring 2004 to continue into the full-year study the following 
academic year (with the exception of Site D, which was dropped from the study because of 
administrative problems in conducting the research). Some teachers decided not to continue, but 
this attrition occurred primarily between different stages of the study and not during the 
experimental interventions (Table 18 in the Technical Notes shows the number of teachers who 
originally volunteered, the number assigned to the two conditions, the number who participated 
in the pilot and the full year, and the number who withdrew from each phase. The Technical 
Notes also provide detailed description of our attrition analyses). 

While we did not find any differences between those teachers who continued and those who 
did not, we do not claim that the CTE teachers who originally volunteered or ultimately 
participated in this research, both as experimental and control, are representative of their 
colleagues. The fact that they volunteered to participate in the research and then continued from 
the pilot study to the full-year study defines them as atypical. As will be discussed in Chapter 7, 
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participants in this study would probably be classified as “early adopters” (E. M. Rogers, 1962, 
1976): teachers who continually seek ways to improve their professional skills and knowledge. 

While our teachers are atypical, this does not mean that their students are. In high schools 
and career centers, students do not choose their teachers. They usually choose CTE courses 
because of an interest in an occupational area, not because of who teaches the classes. Teachers 
who have a good reputation may attract students to their areas, but because of randomization any 
such selectivity in students would be distributed across the experimental and control groups. 

Students. Recruitment of students began by distributing to parents of students in both 
experimental and control classrooms a letter informing them of the nature of the study. They 
were asked to sign and return a form only if they did not want their child to participate (i.e., what 
is commonly called “passive consent”). It is important to note that by opting out, students would 
not participate in data collection. All students in the experimental classes would experience the 
curriculum interventions. Very few parents returned the form. Students whose parents did not 
object to their participation signed assent forms before taking the surveys and tests. To 
encourage their participation and involvement in the data collection, students in both the 
experimental and the control classrooms were given a $10 gift certificate for each of the two 
survey/test administrations (one “pre” and one “post”). 

Implementing the Math-in-CTE Model 

We begin this section by describing how teachers were prepared to implement the Math-in- 
CTE model. We follow with a description of how the professional development was modified for 
the full-year implementation (see Stone et al., 2005, for a more complete discussion of the pilot 
phase of this study) 

Initial Professional Development Strategies. Each of the SLMP replication sites conducted 
professional development workshops. Individuals with expertise in teacher training and 
curriculum integration served as site facilitators and worked with the NRCCTE research team as 
consultants to plan and conduct the professional development workshops at each SLMP 
replication site. 

Workshop facilitators employed several strategies designed to ensure that all experimental 
teachers received the same professional development. For example, common agendas across 
sites were developed for the professional development, including strategies for effective team 
building and guidance on how to work together to raise the mathematics embedded in the CTE 
curriculum to a more explicit level. To further reduce variation in implementation across SLMPs, 
facilitators were provided with a trainer manual that included sample agendas, lesson plan 
templates, and other relevant training materials. 

The purpose of the professional development was to prepare the CTE-math teacher-teams to 
function collaboratively in the experimental effort over all of the SLMP replications. Each SLMP 
was located at a different geographic site, and teacher-teams in each one met together, separate 
from the other SLMPs, to focus on their own occupational area. Within each SLMP, CTE-math 
teacher-teams attended all workshops together, and remained partners throughout the study. All 
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teachers in the workshops received a packet of materials that contained information about the 
study and about their roles and responsibilities as participants. 

Establishing CTE-Math Teacher Partnerships 

As noted earlier, when the CTE teachers applied to participate in the study they were 
required to identify math-teacher partners, preferably from their schools, who were willing to 
work with them throughout the study. The role of the math-teacher partners was to help the CTE 
teachers identify the mathematics in their specific CTE courses, to assist the CTE teachers in 
developing math-enhanced CTE lessons, and to suggest instructional methods to highlight the 
mathematics concepts. Recognizing that a majority of CTE teachers are not formally prepared to 
teach math, and may not have even encountered the math as a college student, the pairing of 
CTE teachers with math-teacher partners was a critical component of the NRCCTE model. 
Importantly, however, the role of the math partners was not to “team teach” or in any way teach 
the math for the CTE teacher; instead, they were asked to provide math support to the CTE 
teacher prior to and after they taught their math-enhanced CTE lessons. 

Building Curriculum Maps 

At the professional development workshops, CTE-math teacher-teams were given basic 
foundational curriculum maps to use as a basis for further revision and identifying the math 
concepts common to their courses, and subsequently, to use as a basis for selecting and 
developing the math-enhanced lessons within their SLMP. These example curriculum maps 
aligned math concepts (e.g., algebra, geometry, trigonometry) with existing high school CTE 
curricula for their specific SLMP. 9 For example, the use of proportions and ratios is critical to the 
preparation of medicines in health occupations. The revised curriculum maps were then used to 
identify and select the math concepts around which they would develop math-enhanced CTE 
lessons. 

Learning the Pedagogy 

Facilitators at each of the workshops introduced the pedagogical framework for developing 
math enhanced CTE lessons — the seven elements — to the teacher-teams (see Figure 3 in Chapter 
2). At the initial professional development workshops, teacher- teams were provided with 
examples of contextualized math lessons gathered from various sources around the country, 
including PISA, and various public teacher resource Web sites. The teacher-teams were 
instructed to include all seven elements in their constructed lesson plans. At the conclusion of the 
workshops, they received copies of the lessons they and their colleagues created and were 
expected to continue working together to develop and refine the lessons for implementation. It is 
important to note that the CTE teachers involved in each SLMP agreed upon a common class 
(e.g., brakes in the auto tech SLMP) to focus on at the beginning of the study. Teacher-teams at 



9 Curriculum mapping is a well-established procedure used by state education agencies (e.g., Colorado and Michigan 
Departments of Education) as well as by nonprofit curriculum development organizations (e.g., V-TECS and 
CoMAP). This is also a strategy used by corporations to identify the academic content of jobs. 
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each of the SLMP replication sites were asked to agree upon, develop, and teach a total of five to 
ten lessons for their SLMP. 

Implementing the Math-Enhanced Lessons 

All teachers were expected to teach all the agreed-upon lessons created for their SLMP. We 
asked the teacher-partners to communicate before and after the CTE teacher taught each math- 
enhanced lesson. Ahead of each lesson, the math teacher provided support for the CTE teacher in 
planning the instruction, answering questions, helping to problem- solve, and offering 
encouragement. Then, following the lesson, the math teacher followed a structured debriefing 
protocol with the CTE teacher and entered reflections onto the NRCCTE Web site or sent a 
written summary via email, fax, or standard mail. This procedure also served as an indicator to 
NRCCTE that the lesson had taken place. 

The NRCCTE research team was sympathetic to the demanding context of high schools and 
understood that implementation of the math-enhanced lessons could not simply be assumed. 
Therefore, participating teachers were asked to agree to the following expectations for 
implementation: 

1. CTE teachers would teach the full set of math-enhanced lessons developed and agreed- 
upon in the professional development workshops for their SLMP class. 

2. CTE teachers would implement the math-enhanced lessons as an integral part of their 
curricula, teaching them where the math naturally occurred as opposed to teaching them 
as stand-alone math lessons or at a convenient time (e.g., at the end of a semester). 

3. Math-teacher partners would provide ongoing support throughout the implementation. 

4. CTE teachers would teach the lessons on their own. 

Teachers’ fulfillment of these expectations was critical to the fidelity of the treatment. 
Therefore, procedures and measures (discussed in Chapter 4) were developed to monitor the 
extent to which the experimental teachers honored their agreement. 

Revising the Math-in-CTE Model 

Building on what we learned during the pilot study, a number of modifications were made to 
our professional development approach for the full year implementation. One change was to 
increase the amount of time devoted to professional development. We began with 5 days in the 
summer of 2004 (SLMP sites were given the option of breaking the 5 days into two segments 
when more convenient for the teachers), 2 days in fall 2004, 2 more in winter 2005, and 1 the 
following summer for debriefing. As in the pilot study, CTE-math teacher-teams attended all 
workshops together, and remained partners throughout the study. Because we had already 
established the effectiveness of the model in the pilot one-semester study, the overarching goal 
of the professional development was to maximize the amount of math that could be enhanced 
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during the course of a yearlong CTE curriculum. It should be noted here that we are talking 
about one CTE class, not a whole CTE program of classes. 

During the summer professional development sessions, CTE-math teacher-teams focused on 
revising their curriculum maps and scope and sequences, and redesigning their lesson plans to 
conform to the revised formats developed by the research team. Building on their work in the 
pilot study, the teacher-teams refined the existing lessons and created new ones using a revised 
template. Priority was given in the workshops to providing time and opportunities for CTE 
teachers to practice and observe others teaching the lessons. The CTE teachers then used a scope 
and sequence chart to position the math-enhanced lessons into their course. This procedure 
helped teachers retain the contextualized nature of the lessons by scheduling them when they 
would naturally occur in their curriculum and increased the possibilities that all lessons would be 
taught throughout the school year. 

Specifically, the workshop objectives were designed to enable the teachers to: 



• Explain the need for “enhanced” mathematics instruction within their respective CTE 
curriculum. 

• Identify the “points of intersection” of mathematics concepts and CTE content within 
their respective CTE curriculum. 

• Develop skills and confidence in creating and implementing math enhanced lessons 
within their respective CTE classroom instruction. 

• Develop specific lesson plans and instructional strategies for enhanced lessons that move 
students from embedded math to related contextual math to traditional math problems. 

• Assess individual student math awareness, knowledge, and skills. 

• Introduce the math enhancement with CTE examples at the embedded level. 

• Reinforce the concept with other examples in the same CTE context as well as in a 
traditional math test format. 

• Assess individual student learning and mastery of math concepts at each of the levels. 

• Continue to integrate the math concepts within daily CTE classroom instruction 
throughout the year. 

• Work consistently and effectively as a member of a CTE-math teacher-team. 

• Fulfill their role and responsibilities in the research process. 



To increase the math support throughout the year, regional math cluster meetings were 
conducted between each of the professional development sessions. Clusters included 2-4 CTE 
teachers from a small geographic area so that it was easier to meet for more focused sessions. 
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Math “captains” selected from the group of math-teacher partners led these groups. Debriefing 
sessions, which included training for the control teachers, were conducted at the conclusion of 
the study in June of 2005. This iterative, reflective, cooperative method of professional 
development is similar to the “lesson study” method that Japanese teachers use (Lewis, Perry, & 
Murata, 2006). 

Data Collection 

A mixed method approach to data collection was employed throughout the study. More 
specifically, this approach is understood as a “concurrent nested strategy” (Creswell, 2003). This 
procedure involves the collection of both quantitative and qualitative data with one given priority 
and the other embedded within the primary method. Miller and Crabtree (1999) give credence to 
using a mixed method approach in evidence-based research design. In reference to randomized 
medical trials, they assert, “Research designs in clinical research inherently require multimethod 
thinking, or critical multiplism, with the particular combinations of data-gathering analysis, and 
interpretation approaches being driven by the research question and the clinical context” (p. 

619). They further highlight the essential interplay of quantitative and qualitative methods in 
evidence-based research: 

The new gold standards . . . need to include qualitative methods along with the 
RCT [randomized clinical trial].... We propose conceptualizing the multimethod 
RCT as a double- stranded helix of DNA. On one strand are qualitative methods 
addressing issues of context, meaning, power, and complexity, and on the other 
are quantitative methods providing measurement and a focused anchor. The two 
strands are connected by the research questions, (p. 613) 

In this study the predominant quantitative method involved pretesting and posttesting of 
students in classrooms and subsequent analysis of the testing data. The pretest provided evidence 
regarding the equivalence of the groups and permitted adjustment for any pretest differences that 
remained despite the randomization. Nested in time between the pretests and posttests was the 
concurrent collection of both quantitative and qualitative data used to document fidelity of the 
treatment and to gain understandings about the teacher experiences during implementation of the 
math-enhancement model. Chapter 4 describes more fully the qualitative methods used to 
measure fidelity of treatment and the results that these methods produced. The following section 
describes our quantitative data collection, the results of which are reported in Chapter 5. 

Liaisons 

In order to keep students’ data anonymous, to minimize teacher influence on student 
participation, and to facilitate simultaneous testing dates at multiple schools, SLMP facilitators 
identified “liaisons” to administer the surveys and tests. Liaisons were frequently counselors, 
other school personnel, or retired teachers. The role of the liaison was to act as a link between the 
researchers and the schools; they were responsible for scheduling test dates and arranging data 
collection. They were also responsible for explaining the study to students and distributing 
consent forms in the control classrooms (experimental teachers were responsible for these tasks 
in their classrooms). Liaisons were also responsible for assigning ID numbers to students and 
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keeping all teacher and student data anonymous and confidential. Each liaison signed a 
confidentiality agreement in order to participate because they were the ones who had the master 
list of student names and ID numbers. For their duties, liaisons were each compensated with a 
stipend (in the full-year study, this was $375 for the first classroom and $100 for each additional 
classroom they tested). 

Data collection in the full-year study was conducted throughout the 2004-2005 academic 
year. In the fall of 2004, experimental and control teachers who continued in the study were 
surveyed prior to the testing of their students. Parent and student consent forms, student surveys, 
and math pretests were administered to students in the experimental and control classrooms. Data 
specific to the fidelity of treatment were collected throughout the year. These data included 
scope and sequence charts, preteaching and postteaching reports, math cluster meeting reports, 
observation data, and associated artifacts (see Chapter 4). At the end of the school year, students 
in the experimental and control classrooms completed surveys and were administered math 
exams and CTE skills tests. Both experimental and control CTE teachers were surveyed, and 
experimental CTE teachers and their math-teacher partners participated in focus groups. 
Debriefing sessions were conducted for control CTE teachers. 

Surveys. Surveys were developed to collect relevant information from the teachers and 
students before the study began. 1 " Teachers were asked about their demographic information and 
their use of math in the classroom (including whether the school or district had asked them to do 
so). After the study ended, teachers were given a postsurvey asking about the effectiveness of the 
model, reporting progress, reflections on its success, and suggested modifications if we were to 
attempt the study again. 

Measuring Academic and Technical Knowledge Achievement 

A total of three different quantitative measures were used to assess students’ mathematical 
performance; each provided information to test a specific hypothesis. To minimize both time 
requirements and burden on participants while still providing a classroom-level average test 
score, only one-third of students in each classroom took each of the three tests (rather than have 
every student take all three tests). To ensure uniformity across all schools and SLMPs involved 
in the study, 40 minutes was established as a time limit for the administration of the math tests 
(pretest and posttests). This minimized the burden on students and teachers by keeping the 
testing time within one class period. The basic measure presented for all tests in this report is the 
percentage of correct answers (correct answers divided by total number of test items). The time 
limit was a design compromise that makes it impossible to compare our results to national 
norms. 

Pretest. Because random assignment was conducted at the teacher rather than the student 
level, pretesting was conducted with the students to determine if the randomized teacher 
assignments had produced groups of students that were equivalent in performance on a 
standardized test of mathematics at the start of the study. The Mathematics section of the 
TerraNova CTBS Survey (CTB/McGraw-Hill, 1997a), a shorter form of the CTBS Basic Battery 



10 Some questions on the surveys were asked for other research purposes not directly relevant to the current study. 
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(CTB/McGraw-Hill, 1997b), was used as a pretest because of the multiple areas of mathematical 
concepts it assesses. Furthermore, it is a traditional, nationally-normed and reliable cognitive test 
of math skills. Level 21/22 was chosen in order to measure high school junior- and senior- level 
mathematics. Although four equivalent forms of the Level 21/22 exam exist, only two forms are 
in print. Of the two printed forms, only Form A was available for use in all states involved in our 
study. (Form C, the other printed form, was embargoed/blocked for use in two of the SLMPs 
since it contains items used on their high-stakes standardized achievement exams.) As a result, 
Level 21/22 Form A of the TerraNova CTBS Survey was selected (McGraw-Hill reports 
reliability coefficients of a = .91 for grade 11 and a = .92 for grade 12.). 

Posttests. For the posttest measures, the students in each classroom were randomly assigned 
to one of three groups corresponding to the three postmeasures of math ability: 

1. The TerraNova CTBS Basic Battery test (CTB/McGraw-Hill, 1997b) was used as a 
traditional measure of mathematical ability ( a = .83 for grade 1 1 and a = .85 for grade 
12). As shown in Appendix B, the test covers a broad scope of mathematics concepts, 
ranging from arithmetic to geometry, statistics, algebra, and problem solving. Commonly 
used as a measure of a student’s mathematical aptitude by secondary institutions, this test 
allowed the research team a snapshot of the test taker’s general mathematical abilities. 

2. WorkKeys® Applied Mathematics Assessment (ACT, 2005), a measure of mathematics 
often used in the workforce, served as our measure of mathematics skills in applied 
contexts. The mathematics on the exam was mostly remedial computation (arithmetic), 
but the exam offered the additional challenge of problem scenarios written out in 
paragraph form. Although many of the problems were not story problems in the true 
sense but rather mathematics that was put into real-life scenarios, students approaching 
the problems may have taken them as story problems such as those seen in traditional 
mathematics classrooms. WorkKeys was originally developed for use in job profiling and 
in most applications generates Level Scores, ranging from 0 to 5 for use in selection and 
promotion of individuals within defined occupational categories. Level Scores and Scale 
Scores are calculated by weighing each for item difficulty, i.e., correct answers on more 
difficult items contribute more to the total score than less difficult items. The more 
precise Applied Mathematics Scale Score, ranging from 65 to 90, employs unequal 
intervals between abilities and rounds all values to the nearest integer. We did not use 
Level Scores or Scale Scores generated by ACT. For consistency of interpretation across 
tests, we scored the items ourselves for percentage correct. 

3. The ACCUPLACER Elementary Algebra test (College Board, 1998), a standardized 
college mathematics placement test used by many colleges and universities around the 
country, was used as a measure of potential postsecondary remediation requirements 
(a = .92 ). The ACCUPLACER test is now generally given on the computer, allowing 
students to jump to more difficult questions when easier questions are answered 
correctly, decreasing the amount of time needed to assess a wide variety of mathematical 
understanding. The paper-and-pencil tests, such as the one utilized in this study for 
consistent administration across participating classrooms, are still used by some schools. 
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The particular test used in this study assessed high school algebra skills and would be 
appropriate as a postsecondary placement test assessing the need for remediation. 

Technical Skills Tests. All students also took the skills or content test for their occupational 
area. We used tests created by National Occupational Competency Testing Institute (NOCTI) for 
the health and IT classrooms. We worked with the Automotive Youth Education Services 
(AYES) for the auto tech tests, and the curriculum consortium MarkED provided a test for the 
business and marketing replication. The agriculture test was a test required of all such programs 
by the agency that administers CTE in the state where that replication was conducted. 

Each technical skill or content knowledge test included but was not limited to the technical 
skills learned in the specific course included in the experiment. The occupational tests were 
given on a different day from the math tests, and for these tests students were allowed the regular 
length of time allotted by the test makers since teachers and students wanted to use the scores for 
other purposes. The use of these tests allowed us to determine whether the experimental group 
experienced any loss of technical skill or content knowledge as a result of spending more class 
time specifically on mathematics, when compared with the control group. 

Math Concepts Covered 

By design, the development of math-enhanced lessons was unique to each SLMP and, 
therefore, the number of lessons taught and the number of concepts within those lessons differed 
across SLMPs. It should be noted that the reason we did not use existing curricula was that, 
simply, none existed which followed our model of moving students from contextual to abstract 
understanding of math concepts. Furthermore, as we discovered in the pilot study, the creation of 
the lessons by the teacher-teams was critical to their buy-in and consequently their engagement 
in the study. The decision to use standardized tests as dependent variables was based on their 
reliability and validity, but this decision caused some of the test items to be out of alignment with 
the concepts taught in the math-enhanced lessons. To assess the degree of alignment of the 
curriculum and the tests, the curriculum in each SLMP was analyzed by a team of math experts 
who checked the content and compiled a list of math concepts addressed. Within the SLMPs, a 
math concept may have been addressed multiple times. The summary of alignment is presented 
in Table 1 (More detail is provided in Tables 18 through 22 in the Technical Notes). 

The number and type of math concepts addressed by the lessons varied widely across 
SLMPs. All SLMPs showed a strong emphasis on number relations and computation, important 
building blocks on which all mathematics content builds. In addition, SLMPs E, C, A, and F’s 
lessons stressed measurement and data analysis. The lessons in SLMPs E, A and B also tended to 
address algebra, with SLMP B adding a problem solving component and SLMP F adding 
trigonometry into the mix. Due to the nature of their occupational content, each SLMP’s 
mathematics tended to gravitate toward a small number of higher-level math areas. SLMP E and 
C’s lessons were most heavily concentrated in the areas of data analysis and algebra, while 
SLMP B emphasized algebra and problem solving, and SLMP F was strong on computation and 
measurement. 
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Table 1 

Math Concepts Addressed by Enhanced Lessons in Each SLMP 



Number of lessons addressing 
each math concept by SLMP 



Math concept 


A 


B 


c 


E 


F 


Number, number relations 


4 


10 


4 


8 


2 


Computation, numerical estimation 


6 


12 


7 


8 


12 


Operation concepts 


1 


0 


0 


0 


0 


Measurement 


3 


0 


7 


5 


12 


Geometry, spatial sense 


0 


0 


1 


0 


2 


Data analysis, statistics, probability 


4 


1 


9 


11 


4 


Patterns, functions, algebra 


3 


5 


1 


7 


2 


Trigonometry 


0 


0 


0 


0 


2 


Problem solving, reasoning 


0 


3 


1 


0 


0 


Communication 


0 


0 


1 


1 


0 



Note. Site D was dropped after the pilot study for administrative reasons. 



The specific items from each of the tests that aligned with the concepts are presented in 
Appendix B. As both tables indicate, the enhanced lessons covered a variety of mathematics 
concepts, ranging from remedial (arithmetic) to more advanced (data analysis, statistics, algebra, 
and trigonometry). This means that the math that is accessible in any given CTE curriculum does 
not easily align with a math textbook or with content-driven state standards. The data in Table 1 
and Appendix B also demonstrate that the kind of math and the amount of math within various 
SLMPs differs substantially. The CTE lessons often contained many math concepts within the 
same lesson, ranging from simple to complex. The result was a spiral of learning, where students 
were given a chance to walk through a review of remedial mathematics before stepping up to 
more advanced material involving geometry, statistics, algebra, or trigonometry. Covering basic 
and advanced mathematics allowed the students to practice their skills as they may be seen in the 
workforce as well as on standardized tests, neither of which allows much margin for even the 
most basic computational error. It is important to note, however, that the sequencing of math 
concepts within an SLMP does not necessarily follow usual math instruction. For example, in 1 
week, students may be addressing a fairly complex set of algebraic equations, the next the CTE 
curriculum may call for instruction in statistics, the next simple geometric calculations. 

The following chapter describes the methods used to assess the degree to which the 
experimental intervention was implemented across the five SLMPs. An overview of the entire 
full-year study design can be found in Appendix L. 
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CHAPTER 4: FIDELITY OF THE TREATMENT 

Throughout the study numerous procedures and measures, both quantitative and qualitative, 
were employed to document implementation of the experimental treatment, to detect variations 
in the treatment that may have influenced student performance on the math exams, and to ensure 
that control teachers conducted “business as usual” in their classrooms. 

Procedures and Measures 

Prior to the start of the full-year study, NRCCTE researchers met with SLMP facilitators to 
review key findings of the pilot study and develop a plan for the full-year implementation of the 
treatment. As a result, a number of procedures were put into place to ensure consistency in 
professional development and in the implementation of the math-enhanced lessons across the 
SLMP replications for the full-year study. These procedures and the measures used to assess 
fidelity of the treatment are described in this section. All measures were aligned to the central 
hypothesis of the study (Miller & Crabtree, 1999), and designed to elicit data that would “build 
explanatory frameworks” (Charmaz, 2000) for the quantitative findings of the pilot study. 

Consistent data collection and careful monitoring of progress at each of the SLMP replication 
sites were central to assuring fidelity of the treatment. An NRCCTE project manager served as 
the primary communicator with the site facilitators and provided oversight for these processes. 
SLMP facilitators worked in concert with NRCCTE researchers to collect data systematically 
throughout the year. All data were submitted to the NRCCTE, where they were entered into a 
database and/or archived for analysis. In turn, the project manager prepared preliminary data sets 
and progress reports to assist the facilitators in monitoring the progress at their SLMP replication 
sites. 

Pretesting Survey of CTE Experimental and Control Teachers 

A survey of all experimental and control-group CTE teachers continuing from the pilot study 
into the full-year study was conducted at the beginning of the 2004-2005 school year, ahead of 
the first student testing period. As reported in Chapter 3, the written survey contained questions 
designed to assess teaching self-efficacy in general, teaching self-efficacy in math, attitudes 
about math, and demographic information. As an ongoing measure of fidelity, teachers were also 
surveyed about their schools’ and districts’ involvement in school improvement initiatives and 
math-related changes they may have made to their courses and curriculum related to math. 

Professional Development 

To assure consistency in the professional development, facilitators at each of the SLMP 
replication sites conducted the same number of workshop days at the same general time of the 
year: 5 days in the summer, 2 days in the late fall, 2 days in late winter/early spring, and 1 day 
for debriefing that summer. Center researchers and SLMP facilitators collaborated in the 
development of a universal agenda (see Appendix C) to guide the substance and delivery of the 
professional development across all the sites. 
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All experimental CTE teachers and their math partners were required to attend all 
professional development sessions. As an additional measure, Center researchers attended and 
observed all sessions, and SLMP facilitators taped the proceedings. The teacher handbook was 
revised for the full-year study to include all changes in research procedures, expectations, and 
contact information, updates to instructional materials, including the lesson template and rubric, 
and all reporting forms needed. To support the professional development, a Web site accessible 
to both facilitators and teachers was also developed as a repository for the lessons and supporting 
instructional materials, including taped lesson demonstrations. The Web site also contained 
announcements, links to resources, and a teacher discussion forum. 

The professional development workshops provided a process and the pedagogic framework 
through which the teachers could develop and implement math-enhanced instructional activities 
(see Chapter 2). However, in the pilot study, we learned that teachers found it difficult to teach 
math-enhanced CTE lessons that they had not themselves written. The teachers noted that their 
success in teaching a math-enhanced lesson hinged on how well the lessons were developed, 
especially in terms of the explicitness of the teacher notes supporting the instruction of the 
embedded math. They also expressed the need for more math instruction associated with the 
practice and demonstration of the lessons in the professional development sessions. Based on this 
finding, a priority was added to the full-year study: assisting the CTE teachers to assure the 
accuracy of their math instruction. 

In the professional development sessions, attention was given to the careful construction and 
review of the seven-element lesson plans in advance of teachers implementing them. Time was 
designated for a thorough critique of each of the lessons. This process focused on correcting any 
deficiencies or errors in the lessons, particularly those related to the instruction of the math 
concepts. In particular, the “teacher notes” sections on the lessons were bolstered to provide 
more “at-hand” information and math support for the teachers at the time of implementation. 

CTE teachers also were given time to practice and demonstrate the lessons in front of their peers 
at the professional development sessions. The math-teacher partners were on-hand during these 
demonstrations to provide explicit math instruction as needed, and to answer math-related 
questions from the teacher group. 

Review of Math- Enhanced Lessons 

While the pedagogic framework remained theoretically consistent with that used in the pilot 
study, each of the seven elements was revamped in the full-year study to explicate the movement 
from the CTE embedded example to related, contextual examples, and finally to the traditional 
examples (see Chapter 2). The seven elements were also refined to ensure that the lessons were 
introduced and concluded as CTE lessons, so that as teachers maximized the math in the lessons 
they did not inadvertently become “math” lessons. Element Seven, which was found to be 
essentially nonexistent in the instruction of the lessons in the pilot study, was reconceptualized as 
“formal assessment” in the revised version. This adjustment was made to ensure that teachers 
would teach the lessons through to the end and assess their students. 

Once these changes were made, researchers revised the lesson plan template and developed a 
rubric containing descriptive criteria for each element (see Appendix D). The purpose of the 
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rubric was to promote more consistency and quality in development of lessons, and to bring 
particular emphasis to the bridging of CTE and math vocabularies. Teachers were required to use 
the lesson plan template in the development of the lesson plans, and to use the rubric to check 
their own progress and provide feedback to others as they demonstrated lessons at the 
professional development sessions. 

SLMP facilitators provided teacher-teams with ongoing feedback and recommendations as 
the math-enhanced CTE lessons were being developed and presented in the workshops. This was 
followed by a review of the lessons by an NRCCTE math expert who corrected any remaining 
mathematical errors. At the end of the study, researchers again reviewed final versions of the 
lessons for accuracy and for final identification and mapping of the math concepts embedded 
within each of the lessons. 

Scope and Sequence Reporting 

At the summer professional development sessions, SLMP facilitators provided the CTE 
teachers with a blank scope and sequence template to promote a more systematic implementation 
of the math-enhanced lessons. Each CTE teacher used the template as a means to position the 
lessons developed within the SLMP into his/her own course that was to be used for testing, and 
subsequently, to develop a schedule for teaching each of the lessons throughout the year. This 
procedure was implemented in response to a major finding in the pilot study indicating that the 
“fit” of the lessons to each teacher’s curriculum and the careful timing of lessons in terms of the 
school year and school-related events were central to the successful implementation of the 
treatment. 

Understanding the nature of schools and assuming adjustments would be inevitable, the 
teachers were asked to update the scope and sequence charts at the fall and winter workshops. In 
this way, the scope and sequence process encouraged the truly contextualized delivery of the 
math-enhanced lessons throughout the year and discouraged the postponing or clustering of 
lessons at the end of a term. SLMP facilitators maintained the master grids and monitored the 
teachers’ progress throughout the year in conjunction with NRCCTE project manager. (See 
Appendix E for an example scope and sequence chart.) 

Math Cluster Meetings 

Regional “math cluster” meetings were conceived in response to pilot study findings in 
which CTE teachers indicated need for more math support during the implementation phase. Led 
by selected math-teacher partners within each SLMP (referred to as “math captains”), the cluster 
meetings provided CTE teachers with math support in the long periods between the professional 
development sessions. The meetings were approximately 2 hours long, and occurred once in the 
fall, once in the winter, and once in the spring, for a total of three meetings during the school 
year. The cluster meetings were organized on the basis of geographic proximity, so that teachers 
could easily access the meetings to acquire needed math support for the upcoming lessons on 
their schedules. They could also debrief the lessons they had taught and share their successes and 
challenges. Attendance was strongly encouraged, but not required if teachers had competing 
responsibilities. The math captains were asked to tape each cluster meeting and report 
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proceedings to the SLMP facilitators who, in turn used the information to provide appropriate 
math support in the upcoming professional development sessions (see Appendix F). This practice 
would provide continuous feedback and support to the CTE teachers while allowing the 
facilitators and researchers to monitor progress and prepare to address common issues across the 
SLMP. 

Preteaching Meetings and Reports 

Each CTE-math teacher-team was asked to meet together within 1 week prior to the teaching 
of each math-enhanced lesson. The purpose of these preteaching meetings was to bolster the 
CTE teachers’ math knowledge and confidence as they prepared for upcoming lessons. This 
procedure reflected a change from the pilot study in which math teachers debriefed the CTE 
teachers following each lesson. The shift to preteaching meetings was instituted as a result of 
teacher feedback at the conclusion of the pilot study in which the experimental CTE teachers and 
their math-teacher partners both indicated that math support immediately preceding each lesson 
was more beneficial than a time of reflection afterward. At the conclusion of each of the 
preteaching meetings, the math teachers filled out a report form, which they submitted to the 
NRCCTE via their SLMP facilitator (see Appendix G). These reports provided evidence that the 
teacher-teams were meeting consistently during the term, and that the CTE teachers were 
preparing to teach each of the lessons as indicated on their scope and sequence charts. The 
reports also provided information from math teachers about how well the lesson was developed 
in terms of integration of the math concepts and the seven-element pedagogic framework. 

Postteaching Reports 

The CTE teachers completed a postteaching report (see Appendix H) at the conclusion of 
each lesson. These reports gathered information from the teachers about their experience in 
teaching the math concepts in the lessons, their confidence in teaching the lessons, their 
perceptions about benefits to their students, and the overall success of the lessons. As a fidelity 
measure, the preteaching reports submitted by the math teachers followed by these CTE teacher 
postteaching reports “sandwiched” each lesson with evidence that the lesson was both planned 
for and taught using the seven element pedagogic framework. 

Observations of Experimental CTE Teachers 

Teaching observations served two purposes: 1) to capture descriptive data about 
implementation of the seven-element pedagogic framework, and 2) to provide supporting 
evidence that teachers implemented the math-enhanced lessons as they were planned. Each 
experimental CTE teacher was asked to submit two videotapes of themselves 
teaching — preferably one from the fall term and one from the spring term. These videotaped 
teaching episodes were supplemented with selected live classroom observations. 

At the onset of the study, the research team determined to have as little impact or influence 
upon the control classrooms as possible in order to avoid the Hawthorne effect wherein 
participants increase desirable behavior because they know they are in a study (McMillan, 2000). 
Similarly, we wanted to avoid any effect of compensatory rivalry (also known as the John Henry 
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effect) in which the control-group teachers may somehow try harder to improve their instruction 
or begin to see themselves in competition with the experimental teachers (McMillan, 2000). 
Therefore, the control classrooms were left to conduct business as usual and were not observed. 
However, researchers conducted in-depth phone interviews of randomly selected CTE 
experimental and control teachers to ascertain the extent to which they were involved in 
enhancing the math in their classrooms. These interviews established an important baseline of 
data at the beginning of the study, which showed that none of the CTE teachers interviewed 
(control or experimental) were already following a systematic method of instruction that utilized 
the seven elements as specified in the Math-in-CTE framework. 

Having determined that the teachers in this study were not previously involved in a 
systematic method for enhancing the math in their courses, and having determined not to disturb 
the control teacher classrooms, the primary purpose of the observations then was to ascertain the 
degree to which the experimental teachers followed the seven elements of the pedagogic 
framework as they taught the lessons. Observers followed a criterion-referenced process using an 
instrument aligned to the seven elements. In other words, the observations were oriented to the 
criteria embedded in the pedagogic process rather than to a counterfactual for which there was no 
basis for comparison. 

Because the observations were criterion-referenced and specifically focused on 
implementation of the seven-element pedagogic framework developed by the NRCCTE, it was 
not possible to utilize a standardized instrument from another study. In the pilot study, expert 
teacher educators and researchers were consulted in the development of a criterion-referenced 
observation tool and processes (Castellano, Stringfield, & Stone, 2003; Center for Applied 
Research and Educational Improvement, 2000). Observers tested the instrument in the pilot study 
and minor changes were made to accommodate revisions to the seven elements in the full-year 
study. A scoring rubric aligned to the seven elements supplemented the observation tool as 
measure of the extent to which a lesson was taught (see Appendix I). 

Two researchers, both of whom conducted observations in the pilot study, were selected to 
observe in the full-year study. The observers conducted “focused and selective observations” 
(Angrosino & Mays de Perez, 2000) of both live and videotaped lessons using the seven-element 
math-enhanced lesson plans as the reference. They scripted evidence that supported the teaching 
of each element as it was planned and the extent to which each was addressed. At the conclusion 
of each observation, they used the scripted evidence to complete the scoring rubric and assign a 
composite score for each lesson. In addition, observers commented on the context for the lesson, 
the teaching strategies used, barriers to the lesson, math-related errors, unexpected classroom 
occurrences (e.g. fire drills), and other anecdotes. Observations were not focused on teaching 
quality or efficacy, or on the students and their behaviors. The observers trained together in two 
sessions, for a total of approximately 8 hours, until they reached agreement in scoring prior to 
viewing the videos and conducting the live observations. At the conclusion of the study, they 
conducted a formal reliability check (see Table 8). 

Student Work Samples 
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The CTE experimental teachers were asked to submit one sample of student work per lesson 
taught. The work sample was to be representative of a formal assessment used to fulfill element 
seven of the lesson plan. The purpose for collecting the assessment samples was to provide a 
supporting source of evidence that the teachers had instructed the lessons through to their 
conclusion. This measure was put into place as a result of inconclusive findings in the pilot study 
related to the extent to which students were assessed on the math concepts in the lessons and/or 
that the lessons were completed. 

Focus Groups of CTE Experimental and Math Teachers 

Researchers from the NRCCTE conducted teacher focus groups at the end of the study in 
conjunction with the debriefing sessions at each of the SLMP replication sites. The purpose of 
the focus groups was to elicit in-depth descriptions of the process and the pedagogy as the 
teachers experienced it, that is, to more clearly understand what worked and what didn’t work 
from their perspectives. To guide the focus groups, a “question path” (Krueger, 1998; Krueger & 
Casey, 2000) was developed that moved from a general question about the overall experience of 
participating in the study to more specific questions about the implementation of the model and 
what made it work (see Appendix J). With one exception, CTE-teacher and math-teacher focus 
groups were conducted separately so that participants would feel more at-ease responding 
candidly about their experiences. At SLMP F, logistics dictated that math and CTE teachers join 
together as one group. Participation in the focus groups was voluntary; sessions were audiotaped 
and transcribed for analysis. 

Poststudy Surveys of Experimental and Control Teachers 

CTE teachers participating in the experimental and control groups were surveyed at the 
conclusion of the pilot study. Experimental teachers were asked both quantitative and qualitative 
questions related to aspects of the professional development process, the effectiveness of the 
pedagogic framework, their implementation of the lessons, the impact of the lessons on students, 
and the impact of the study on their own attitudes and teaching. As a measure of fidelity, both 
sets of teachers were again asked about math-related activities occurring throughout the year that 
may have impacted the study, including their schools’ and districts’ involvement in school 
improvement initiatives, math-related changes they may have made to curriculum and courses, 
and their own involvement in professional development outside the scope of this study. 

Analysis of Fidelity Data 

As discussed earlier, we employed a mixed methods approach that required appropriate use 
of both quantitative and qualitative research procedures and strategies. The predominant 
quantitative approach of this study was the experimental design that involved random assignment 
of classrooms and the analysis of student test data, the results of which are reported in Chapter 5. 
The qualitative approach we used to give description to these quantitative findings was primarily 
that of grounded theory, in which one explores processes, activities, and events for the purpose 
of building theory (Charmaz, 2000; Creswell, 2003; Glaser & Strauss, 1999). 
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For the purpose of assessing fidelity of the treatment our goal was to provide an accurate or 
credible description of the treatment as it occurred and to present “conclusions that are 
believable and trustworthy” (McMillan, 2000). That accuracy was accomplished primary through 
“triangulation” of data from the quantitative and qualitative measures we described in the 
preceding section. The analyses frequently required data transformation, that is, the codification 
of qualitative data to provide a numeric representation and, conversely, the qualification of 
quantitative data to support thematic development (Creswell, 2003; Caracelli & Green, 1993). 

For instance, observation scripts were coded and results were articulated in numeric form. 
Responses from surveys were articulated in simple percentages or mean scores to support 
thematic descriptions. The evidence from these multiple sources of data were also used to “build 
a coherent justification for the themes” (Creswell, 2003) that are presented in the “What Works” 
section at the end of the next chapter. 



Findings 

Overall, we found that the procedures employed by the NRCCTE researchers and SLMP 
facilitators were successful in assuring fidelity of the treatment. The professional development 
and math cluster meetings were delivered with consistency across the SLMP replication sites. 
While some variations existed, the evidence drawn from multiple sources, both quantitative and 
qualitative, revealed that CTE teachers across the SLMP replications implemented the math- 
enhanced lessons as planned most of the time. The following sections describe these findings in 
more detail. 

Consistency in the Professional Development 

There were no notable variations across the SLMP replications either in the amount of 
professional development rendered or in the substance of the workshops. Facilitators followed 
the procedures and conducted professional development at the time of year and for the number of 
days agreed upon at the onset of the full-year study. They submitted professional development 
agendas to the NRCCTE for review and approval in advance of each workshop. They also 
videotaped each workshop to document the professional development process. Additionally, at 
least one NRCCTE researcher attended and observed each of the workshops. Through these 
procedures, researchers found the workshops across the SLMP replications to be consistent with 
requirements outlined in the universal agendas. 

Workshop attendance was required of all CTE experimental teachers and their math-teacher 
partners. There was no indication of abuse of the attendance policies. Facilitators at all 
replications reported perfect attendance at the summer workshops. At SLMP E, facilitators 
conducted a second, make-up workshop in response to a number of scheduling conflicts with the 
teachers’ summer schedules. The few absences reported in the fall and winter workshops 
conducted during the school year were excused. At SLMP C, one absence was reported in the 
fall, and two in the winter. Two absences occurred at SLMP F in the fall workshop. SLMP A and 
SLMP B reported perfect attendance at all workshops. 

The experimental CTE teachers’ overall rating of the effectiveness of the professional 
development was consistently favorable. The posttest survey requested CTE teachers to rate the 
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effectiveness of professional development sessions on a scale of one to four, 1 meaning “very 
effective” and 4 meaning “not at all effective.” Overall, the CTE teachers rated the effectiveness 
of professional development sessions with an average score of 1.4, a finding confirmed in the 
focus group interviews. In the case of the latter, both the CTE and math experimental teachers 
and teachers expressed satisfaction with the professional development, describing it as a 
“worthwhile” experience and a good use of their time. 

SLMP facilitators were also responsible for the consistent delivery of math cluster meetings. 
All experimental CTE teachers were provided access to these meetings at locations in close 
proximity to their schools, with the exception of SLMP B, where a few individual teachers were 
separated by long distances. In these unique cases, the SLMP facilitator and selected math 
captains provided support by telephone. Proceedings of the math cluster meetings were 
documented by videotape whenever possible. Math captains also submitted reports to SLMP 
facilitators, who in turn addressed specific issues to the whole group at the professional 
development sessions. 

As noted in the previous section, attendance at the cluster meetings was strongly encouraged, 
but not required. There was considerable variability in the reporting of cluster meeting 
attendance across SLMPs, however, facilitator reports indicated an overall attendance of around 
60% in the fall, 70% in the winter, and 50% in the spring meetings. In cases where teachers 
could not attend, math captains provided teachers with meeting notes and/or access to the 
meeting video. 

The math cluster meetings received mixed reviews in terms of their effectiveness in 
preparing the teachers to teach the math concepts in the lessons. On the poststudy survey, the 
teachers’ ratings of effectiveness varied greatly across the SLMPs as indicated in Table 2. 

Table 2 

CTE Teachers’ Ratings of the Effectiveness of Math Cluster Meetings 

SLMP 

A B C E F All 

Mean perceived effectiveness of 

preparing CTE teachers to teach N/A 1.5 2.4 2.9 2.2 2.1 

the math concepts 

% of CTE teachers 

recommending the meetings in N/A 81.8 55.0 11.1 25.0 44.4 

future implementations 

Note. Judgments were made on a 4-point scale (1 = very effective, 4 = not at alt effective). Site D was dropped after the pilot 
study for administrative reasons. 

This mixed review of the cluster meetings was echoed in the focus groups where the CTE 
teachers described them as helpful, but in the end, not essential to their success in implementing 
the lessons. Teachers reported that conversations at the cluster meetings were often focused on 
more generalized concerns, such as scheduling difficulties, and the amount of time it took to 
teach the lessons. Most suggested the meetings were needed less in the full-year study because of 
teachers’ familiarity with the lessons and ready access to a math-teacher partner. A few 
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individuals in the focus groups complained that the meetings took too much of their time. This, 
however, does not negate those who expressed gratitude for the much-needed support and 
collegiality, and others who described how the meetings helped them become more confident in 
their math abilities. 

Variations in Development of the Math-Enhanced Lessons 

The most evident variation in treatment across the SLMP replications was found in the 
numbers of lessons developed for each SLMP and, thus, in the number and type of math 
concepts addressed within those lessons (see Chapter 3 for a discussion of the variation in math 
concepts). This variation was anticipated as an outcome of a main premise of the study: that we 
enhance the math already embedded within the CTE curricula as opposed to overlaying math 
concepts onto the curricula. Because we based the math-enhancement process on this premise, 
differences across SLMP content dictated the math concepts that were identified and mapped, 
and subsequently, in the topics and numbers of lessons that were developed at each SLMP 
replication site. 

In the summer professional development session before the full-year study, SLMP facilitators 
at each site led teachers in a critique of the math-enhanced lessons developed in the pilot study. 
They also provided opportunities for teacher-teams to identify and map new math concepts, and 
develop new lessons. On the whole, teachers in each of the replications expressed the desire to 
improve and expand upon the set of lessons they developed in the pilot study. Having taught 
them once within a semester’s time, the CTE teachers were especially anxious to place them into 
their curriculum where they fit best over a year’s worth of instruction. They wanted to try them 
again — to teach them with improvements, e.g., with more handouts for different ability-levels of 
students, with specific vocabulary for bridging the languages of math and CTE, and with better 
methods for assessments, especially in elements two and seven (see Chapter 2 to review the 
seven elements). Therefore, few lessons from the pilot study were eliminated, and few new 
lessons were developed. The most common occurrence was the overhaul of lessons that were 
either underdeveloped or too redundant. Lor instance, some teacher-teams split up lessons that 
were too lengthy or held too many math concepts. In other cases, lessons were added to expand 
upon particular concept introduced in earlier lessons. Across the study, the teachers developed 
more instructional materials and assessments to address their students’ varying math abilities and 
to promote the “bridging” and “maximizing” of math concepts within the lessons. 

As in the pilot study, teachers in four of the five SLMPs agreed on a common set of lessons 
to teach (see Table 3). The exception was SLMP E, where the content was exceptionally broad 
and fewer commonalities existed in curricula and courses. Here, the teacher-teams shared their 
lessons with one another, but worked separately to create lessons unique to each individual CTE 
teacher’s curriculum. This condition required more individuality in the development of the 
lessons, resulting in a higher total of lessons developed within the SLMP, but fewer taught by 
each teacher. Many of the individualized lessons in SLMP E tended to be longer lessons that 
addressed multiple math concepts. 
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Table 3 














Lesson Implementation by Teachers 












SLMP 




A 


B 


c 


E 


F 


All 


Number of math-enhanced 
lessons teachers developed 
and agreed to teach 


7 


15 


11 


6 a 


17 


56 


% of teachers who reported 
teaching all agreed-upon 
lessons 


100 


100 


95.2 


66.7 


78.6 


87.7 


% of teachers who reported 
not teaching all agreed- 
upon lessons 


0.0 


0.0 


0.0 


33.3 


7.1 


7.0 


% of teachers not 
responding 


0.0 


0.0 


4.8 


0.0 


14.3 


5.3 



Note. a At SLMP E, a total of 25 math-enhanced lessons were developed, however each teacher taught an average of six math- 
enhanced lessons adapted to his/her own curriculum. These math-enhanced lessons addressed multiple math concepts and were 
generally longer than one class period, some lasting several days. Site D was dropped after the pilot study for administrative 
reasons. 



The small number of lessons developed in SLMP A was attributed to attrition of teachers. 
Already a small replication in terms of numbers of teachers, there were significantly fewer 
teachers to develop lessons for the full-year study. Teachers at SLMP B and SLMP F not only 
agreed upon a common set of lessons, but identified a recommended order for teaching the 
lessons in which one would build upon the next. Based on their experiences in the pilot study, 
they perceived a sequence of courses would strengthen the students’ understandings of both the 
CTE and the math concepts. This sequencing was possible because many of the schools and 
programs within these SLMPs followed the same or similar curricula. 

The Extent to Which Lessons Were Implemented 

Central to assuring fidelity of the treatment was ascertaining the extent to which teachers 
taught the lessons. Poststudy survey data presented in Table 3 indicated that 88% of the teachers 
across the study reported teaching all of the agreed-upon lessons in the single course used for 
testing. However, the variations in the table require explanation. For example, in SLMP F, 
teachers attempted 17 lessons, which they agreed was a rigorous undertaking. While their 
percentage of completion was lower, they still taught an average of 15 lessons, which was equal 
to SLMP B, and more than the number reported at the other sites. At SLMP E, teachers 
developed a total of 25 lessons for their SLMP, but adapted an average of six of these for use in 
their own curriculum. 
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A more accurate and detailed account of the teaching activity was found through examination 
of the preteaching reports submitted by the math teachers, and the postteaching reports and 
student work samples submitted by the CTE teachers. Table 4 provides a more complete picture 
of the teaching activity based on the total number of classroom interventions (math-enhanced 
lessons) that were slated to have been taught in each of the SLMP replications. We found that, 
overall, the math-enhanced lessons were taught somewhere between 86-92% of the time, 
depending on the data set examined in this table. A comparison of the preteaching and 
postteaching reports indicates that 97% of the lessons were reported by the math teachers as 
having been reviewed for teaching, and that 92% of the lessons were subsequently reported by 
the CTE teachers as having been taught. Student work samples provided the most compelling 
evidence that the teachers were engaged in teaching the lessons, however, the actual collection of 
the samples was inconsistent across the replications, resulting in the reported 86%, which is 
lower than the preteaching and postteaching reports indicated. Of note were SLMP E and SLMP 
F, where the lower percentages may have been attributable, in part, to project-based assessments 
that were more difficult to identify and submit. 

Table 4 

The Extent to Which Math-Enhanced Lessons Were Taught , Based on Preteaching and 
Postteaching Reports and Student Work Samples 



SLMP 





A 


B 


C 


E 


F 


All 


Total number of math- 
enhanced lessons scheduled 
for teaching 


14 


166 


231 


61 


238 


710 


Percentage of the math- 
enhanced lessons reviewed in 
preteaching meetings 


100 


93 


100 


93 


99 


97 


Percentage of math-enhanced 
lessons reported as completed 
in the postteaching reports 


93 


93 


90 


84 


95 


92 


Percentage of math-enhanced 
lessons evidenced by student 


100 


98 


97 


79 


69 


86 



work samples 

Note. Site D was dropped after the pilot study for administrative reasons. 



With regard to the scope and sequence planning, we asked teachers the extent to which they 
were able to teach the lessons as they were originally scheduled or sequenced (see Table 5). On a 
scale of one to five, where 1 indicated all lessons were taught as sequenced and scheduled and 5 
indicated that none of the lessons were taught as sequenced and scheduled, the average score 
across the study was 2.1, suggesting that most of the lessons were taught as sequenced and 
scheduled. There were some SLMP-specific variations in this response, however. 
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Table 5 

Experimental CTE Teacher Report of Fidelity to Lesson Implementation Plan 


SLMP 




A 


B 


C E F 


All 


Taught math-enhanced 
lessons as originally 
sequenced and scheduled 

m 


2.0 


1.6 


2.7 2.6 1.3 


2.1 



Note. Reports were made on a 5-point scale (1 = All of the lessons were taught as sequenced and scheduled, 5 = None of the 
lessons were taught as originally sequenced and scheduled). Site D was dropped after the pilot study for administrative reasons. 



Of note were the better scores at SLMP B and SLMP F, which may have been a function of 
the voluntary sequencing of their lessons, as reported in the previous section. 

While we asked the CTE teachers to indicate the amount of time dedicated to each of the 
lessons on each of their postteaching reports, analyses of this data proved inconclusive. As in the 
pilot study, we found that a math-enhanced lesson was not necessarily equal to the teaching of 
one math concept, nor was it equivalent to one class period. If successfully followed, the teachers 
taught the lessons as CTE lessons within the context of their classrooms, shops, and laboratories. 
For instance, a math-enhanced lesson on sprayer calibration was observed to take place during 1 
week of instruction in a school where students were using a tractor in a field. In another school, 
the same lesson was conducted over 2 days in a classroom shop area using a hand-sprayer. In 
another example, a teacher with special education students chose to extend a lesson across 
several class periods to reinforce the math concepts that another teacher was able to address in 
one class period. In essence, the inability to extract time as an accurate measure of the treatment 
was a function of the truly contextualized nature of the model. 

The Extent to Which the Seven-Element Pedagogic Framework was Implemented 

Poststudy teacher survey data indicated that teachers considered the pedagogic framework to 
be “very effective.” This finding was underscored in the focus groups, where CTE and math 
teachers widely expressed their satisfaction with the seven-element framework. However, the 
CTE teachers were also candid about their need to tweak the lessons to fit their particular 
situation, e.g., to accommodate the length of class period, to fit in with a particular activity or 
project, to more closely fulfill a required curriculum standard, etc. This finding was supported in 
the survey where the CTE teachers, on the average, reported modifying or changing from a few 
to half of lessons before teaching them (see Table 6). 
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Table 6 



The Extent to Which Experimental CTE Teachers Reported Modifying the Math- 
Enhanced Lessons 



SLMP 


A 


B 


C E 


F 


All 


Mean report of lesson modification 3.0 


3.5 


3.5 3.6 


4.5 


3.7 



Note. Responses were made on a range of 1 to 5 ( 1 = Modified or changed every lesson, 5 = Did not modify or change any of the 
lessons). Site D was dropped after the pilot study for administrative reasons. 



Teachers described a wide range of modifications to the lessons such as: making 
adjustments/corrections to the CTE content; adding worksheets; accommodating special needs 
students; improving/adding math examples; creating PowerPoint slides; adding technology; 
correcting typos, etc. While we know from these data that teachers were modifying the lessons, 
the changes they made did not necessarily negatively impact the seven-element pedagogic 
framework. Therefore, to ascertain the extent to which the seven elements were addressed, we 
also examined the preteaching and postteaching reports. 

In the preteaching reports, we asked the math teachers to indicate if all seven elements were 
clearly represented in the lesson the CTE teacher planned to teach. In the postteaching reports, 
we asked the CTE teachers to indicate if they were able to teach all seven elements in the lesson. 
Data from these reports indicated the lessons were reviewed and in good form ahead of the 
lessons 95% of the time, and that the CTE teachers, by self-report, successfully taught all seven 
elements 95% of the time (see Table 7). 

Table 7 

The Extent to Which All Seven Math-in-CTE Elements Were Addressed in the Lessons, as 
Indicated in the Preteaching and Postteaching Reports 



SLMP 





A 


B 


C 


E 


F 


All 


% preteaching reports 


100 


89 


98 


93 


96 


95 


% postteaching reports 


100 


99 


94 


78 


97 


95 



Note. Site D was dropped after the pilot study for administrative reasons. 



To further support this finding, we observed 88 videotaped teaching episodes and conducted 
17 live observations. In total, we analyzed approximately 70 hours of teaching to determine the 
extent to which the teachers were observed to have addressed each of the seven elements in their 
lessons according to the rubric criteria. On a 5-point scale where a score of 5 indicated that all 
criteria were met, the average composite teaching score across the SLMPs was 4.1. Separate 
analysis of video and live teaching episodes showed relatively no difference in the composite 
scoring. This meant that teachers across the study were successful in addressing most of the 
criteria representing the seven elements in their lessons (see Table 8). 
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Table 8 

Average Composite Teaching Scores Derived From Observation of Video and Live Teaching 
Episodes 



SLMP 





A 


B 


C 


E 


F 


All 


Number of video & live 














observations 


5 


22 


43 


12 


18 


100 


M composite teaching score 


4.2 


4.4 


3.8 


4.0 


4.4 


4.1 a 



Note. Observations were rated on a 5-point scale (1 = no observable criteria were met, 5 = all observable criteria were met). Site 
D was dropped after the pilot study for administrative reasons. 

“reliability coefficient = .93. 



We were challenged both by inconsistent submission of the tapes and by the brevity of some 
of the teaching episodes; fewer than half represented a complete lesson. (Note: A complete 
lesson covered elements one through six; we did not expect to observe element seven, formal 
assessment, which we assume to have taken place at a different time.) Additionally, a number of 
the live observations were also partial lessons. There were some legitimate reasons for this 
occurrence, primarily that a number of the lessons were created to be taught over several days. In 
other cases, lessons were associated with projects/assignments in laboratories, shops, or other 
locations outside the classrooms that were not easily recorded. In this sense, the tapes accurately 
portrayed the contextualized nature of the lessons. When we looked only at the composite scores 
for full lessons, the mean score increased to 4.5. In sum, the composite scoring of the 
observations lent support to findings from the preteaching and postteaching reports that the seven 
elements were successfully followed most of the time. 

The student assessment samples submitted by the teachers provided substantial evidence of 
the implementation of the lessons, but proved less conclusive as specific evidence of element 
seven, formal assessment. While it was clear that students were being assessed, about 32% of the 
samples collected were clearly identifiable as formal assessments specifically related to element 
seven. The other 68% of the samples were graded worksheets most of which were originally 
developed to support the instruction of the lesson rather than assess students’ understanding. 
While it appeared that teachers might have utilized the worksheets as formal assessments, we 
had no evidence to support this claim. 

Math-Related Influences 

Of particular concern throughout the study was the potential for uncontrolled, external math- 
related influences to impact the study. Theoretically, such concerns were addressed in the 
experimental design itself through random assignment of the classrooms. However, we were 
cognizant of math-related activities occurring in some of the schools and districts of participating 
teachers and have attempted here to report it as a matter of fidelity. 

Prior to the start of the pilot study, we conducted interviews with CTE experimental and 
control-group teachers and found no evidence that the CTE teachers were engaged in any 
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systematic approach similar to the treatment in this study. While most of the CTE teachers 
reported addressing the math in their courses in some way, they were not engaged in the 
identification and mapping of math concepts within CTE curriculum and/or subsequent 
development of math-enhanced CTE lessons using anything similar to the Math-in-CTE seven 
elements. We found the most common approach to be a cursory walk-through of the math in 
lessons, projects, or problem-solving scenarios. 

In the preteaching and postteaching surveys in the full year study, control-group and 
experimental CTE teachers were asked to indicate if math-related activities occurred at their 
districts/schools, and if course-level changes were made as a result of district/school mandates. 
We also asked the control teachers to indicate if they had made any math-related changes to the 
courses used for the study (see Table 9). 

Table 9 



Math-Related Influences in the Pilot and Full-Year Studies, as Reported on the Preteaching and 
Postteaching Surveys 





Pilot study 


Full-year study 




% 

control 
teachers 
(n = 74) 


% 

experimental 
teachers 
{n = 57) 


% 

control 
teachers 
{n = 74) 


% 

experimental 
teachers 
{n = 57) 


District/school made math- 
related changes to the course to 
align with other standards/ 
testing 


19.6 


34 


19.1 


20.8 


Teacher made math-related 
changes to the course to align 
with other standards/testing 


2.7 


N/A 


5.4 


N/A 



Note. Ns are the same for pilot and full-year studies because only teachers who continued to the full year were asked about the 
previous (pilot) year’s changes. Ns shown are higher than denominators used in percentage calculations due to nonresponses not 
being included in the percentage calculations. Teachers in our experimental group were asked to make changes for the purposes 
of the study, so this question is not applicable (N/A) to them. 



Overall, we found no significance in the distribution of the reported math-related activities 
between the control and experimental groups. Both control and experimental teachers at SLMP E 
and SLMP C reported more math-related activity at their school-and district-level than teachers 
at the other sites did. In the focus groups, they attributed this activity primarily to state-level 
mandates being implemented in their schools. Reports of control teachers having made changes 
to their courses were infrequent: two in the pilot study and four in the full-year study. 

In the following chapter (Chapter 5), we describe the results of the quantitative analyses. 
Chapter 6 presents the qualitative results. 
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CHAPTER 5: WHAT WE FOUND: QUANTITATIVE RESULTS 

The purpose of the study was to determine whether the experimental intervention was 
sufficiently powerful to affect CTE student performance on standardized tests of mathematical 
achievement. After the intervention, which comprised an academic year of enhanced 
mathematics instruction, three different measures were used to assess math performance. Within 
each classroom, experimental and control, students were randomly assigned to take one of three 
tests: the TerraNova CTBS Basic Battery, the ACCUPLACER Elementary Algebra test, or the 
WorkKeys Applied Mathematics Assessment. This chapter examines student performance on 
these three tests following the intervention. The chapter also presents results from tests of 
occupational skills and knowledge, which we administered to determine whether the time spent 
on enhancing mathematics, had an effect on the acquisition of occupational skills or content 
knowledge. 



Impact of Enhanced-Math-in-CTE Classrooms 

As explained in Chapter 3, we administered a pretest measure of mathematical achievement, 
the TerraNova CTBS Survey to all students prior to the intervention to test the similarity of the 
experimental and control groups and to allow statistical controls if they were not equivalent (see 
Appendix K). While the groups were statistically equivalent overall, some pretest differences 
existed between groups in individual SLMPs. For this reason, as well as to follow conservative 
guidelines (Fraenkel & Wallen, 2003), we decided to include pretest scores as a covariate in all 
of our analyses. 

Because the treatment was administered at the classroom level, we were interested in testing 
classroom level effects. However, the measures were collected at the student level. Because 
students within a class share a similar context, they are more similar to each other than to 
students in other classrooms. Therefore, students as unit of analysis are not statistically 
independent; using statistical tests that assume independence would inflate Type I error rates 
(Murray, 1998). The best way to avoid this problem is to use a hierarchical analysis technique 
that accounts for the nested design and therefore each level of variation. We used hierarchical 
linear modeling (HLM) software (Bryk & Raudenbush, 1992) for our analysis. Student math 
performance data were entered in Level 1 and classroom effects (experimental vs. control group) 
were modeled at Level 2. In laymen’s terms, we used classroom means, rather than individual 
student scores, as our unit of analysis. For more information on this statistical technique, we refer 
the reader to various resources (Bryk & Raudenbush, 1992; Murray, 1998; Snijders & Bosker, 
1999). 



Overall Comparisons 

Our first test of the intervention effect was to compare all experimental SLMPs to all control 
SLMPs. Three separate HLMs were run — one for each of the posttests. Participants were 
excluded only if they could not be assigned to a classroom, or if they had not taken either the 
pretest or posttest. Only one classroom is included for any teacher. Each HLM was conducted as 
follows: 
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Level- 1 Model 

Posttest = p 0/ + P, ; *(pretest)ij + r {j 

Level-2 Model 

Po j = Too + Yoi*( Condition ) ; + u 0j 

Pi j : Yio 

Since pretest scores were centered around the overall mean, P, v represents the classroom 
mean score on the Posttest for a student in each classroom j with average achievement on the 
pretest. In this model, Posttest score consists of the group mean (J3 Q/ ) and the overall effect of the 
pretest ( J3 1; ) as well as random error (r tJ ). The group mean (P Q/ ) itself is assumed to consist of the 
overall grand mean (y 00 ) as well as the effect of being in the experimental group (y 01 ) and random 
error (m 0; ). Because y 01 is coded as “0” for the control group, y 00 reflects the expected score for a 
student in the control group with an average pretest score. The effect of the pretest score ( |3 1; ) on 
the posttest score is presumed to be constant across all classrooms. Results for each of the three 
posttests are presented below. 

TerraNova 

Lor the TerraNova analysis 136 classrooms were used (57 experimental, 79 control), which 
represented 591 students. The results in Table 10 indicate that y ()0 , y 01 and y 1() are all significant. 
y 01 is the coefficient of interest. We expected that pretest scores would be strongly related to 
posttest scores, so we were not surprised that y 10 does not equal zero. Critical, however, was the 
treatment group effect as indicated by y 01 . TerraNova scores are positively affected both by 
pretest scores and by the treatment. According to the model shown in Table 10, a student from a 
control classroom with average scores on the pretest would on average answer 45% of the 
TerraNova items correctly. A comparable student from the experimental group would have a 
score about 4% higher, or 49%. This is a moderate experimental effect according to Cohen (see 
effect size discussion following the WorkKeys results). 

Residual variance components are shown in Table 1 1 for the Terra Nova model both before 
and after the effect of the experimental condition was included. The size of Tau in the base 
model, relative to Sigma squared, indicates that 14% of the variation in student posttest scores is 
due to classroom- level, rather than student-level, effects [24.8 / (24.8 + 152.6) = .14]. The drop 
in Tau from the base model to the full model at level 2 tells us that 13% of the variation in 
classrooms [(24.78 - 21.47) / 24.78 = .13] can be accounted for by the experimental condition. 
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Table 10 



Hierarchical Lineal Model Fixed Effects, With TerraNova as Posttest 



Predictor 


% correct 


df 


P 


t 


Cohen’s d 
(Effect size) 


y 00 - expected posttest score 


45.353305 


128 


<0.001 






y 01 - experimental effect 


4.109598 


128 


0.003 


3.111 


0.550 


y l0 - pretest effect 


0.642262 


549 


< 0.001 


17.785 


1.518 



Note. y 00 = Posttest score for student with pretest score at the grand mean for control group. 
y 01 = Increase in posttest score for student in experimental group. 

y 10 = Increase in posttest score for each point above the grand mean scored by a control group student. 



Table 11 



Final Estimations of Variance Components for TerraNova 





Variance component 


SD 


df 


f 


P 


Variance within 
classrooms 


cr = 152.598 


12.353 








Base variance 
between classrooms 


T= 24.781 


4.978 


128 


211.696 


<0.001 


Full variance between 
classrooms 


T= 21.470 


4.633 


127 


197.137 


<0.001 



Although most of the variation in TerraNova scores is not attributable to differences between 
classrooms, being in the experimental group is correlated with more than a 4% increase in 
TerraNova scores, which is certainly substantial from a pedagogical standpoint. The remaining 
variance is due to the many other student-level and teacher-level factors that influence test 
scores. These likely include factors such as teacher ability and student prior knowledge and 
motivation. Although many factors determine a student’s test score, being in the experimental 
group in this study is associated with a significant increase in student test scores. 
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ACCUPLACER 

For the ACCUPLACER analysis 129 classrooms were used, which represented 571 
students." Again, we were interested in how being in the treatment group effects test scores, as 
demonstrated by y 01 . ACCUPLACER scores are positively affected both by pretest scores and by 
the treatment. 



Table 12 

Hierarchical Lineal Model Fixed Effects, With ACCUPLACER as Posttest 



Predictor 


Coefficient 


df 


P 


t 


Cohen’s d 


Too 


38.955338 


129 


< 0.001 






Yot 


2.950693 


129 


0.018 


2.400 


0.423 


Yio 


0.596972 


568 


< 0.001 


17.380 


1.458 



Note. y 00 = posttest score for a student with pretest score at the grand mean in control group. y 01 = increase in posttest score for a 
student in experimental group. y 10 = increase in posttest score for each point above the grand mean scored by a control group 
student. 



According to this model, a classroom in the control group with average scores on the pretest 
would have a mean ACCUPLACER posttest score of 39% (Table 12). For a class in the 
experimental group, the predicted score would be about 42% — almost 3% higher. The effect size 
of 0.423 indicates a moderate effect of being in the experimental group. 

Table 13 shows that classroom level variables account for 7.5% of the variance in student 
scores on the ACCUPLACER, and that membership in the experimental group accounts for 10% 
of the variance at the classroom level (see formulas in TerraNova section above). 

Table 13 



Final Estimations of Variance Components for ACCUPLACER 



Final estimation of variance 


Variance 


SD 


df 


x 2 


P 


components 


component 










Variance within classrooms 


a 2 = 159.562 


12.631 








Base variance between classrooms 


T= 12.957 


3.599 


128 


174.553 


0.004 


Full model variance between 
classrooms 


x= 11.596 


3.405 


127 


167.471 


0.009 


R 2 


0.10 











11 TerraNova data have a larger n than ACCUPLACER or WorkKeys due to some missing data for those tests. 



48 



National Research Center for Career and Technical Education 




Building Academic Skills In Context: Testing the Value of Enhanced Math Learning in CTE 



WorkKeys 

For the WorkKeys analysis 126 classrooms were used, which represented 536 students. 
According to these results, WorkKeys scores are of course dependent on pretest scores, but 
unlike the previous tests, not on whether or not the classrooms were in the experimental group 
(Table 14). According to this model, a student with a pretest score at the grand mean would be 
expected to answer 59% of the WorkKeys items correctly, an average higher than for either the 
TerraNova or ACCUPLACER. For each additional point a student received on the pretest, their 
expected WorkKeys score would increase less than 1%. While the mean scores for the 
experimental classrooms are higher those for the control classrooms, the difference is not 
significant. The inability of the WorkKeys test to detect an effect may be due to a lower level of 
math questions, as indicated by the higher scores (percentage correct) for both groups on this test 
compared to the other two, or that the kind of math tested by WorkKeys was not addressed in the 
various replication sites. 

Ten percent of the variance in student scores on WorkKeys is attributable to classroom level 
effects; however, virtually none of the variance in classroom scores is due to being in the 
experimental group (see Table 15). In fact, because the model variance between classrooms is 
larger than the base variance, indicating that the variable indicating experimental or control 
condition functions as a random effect (Snijders & Bosker, 1999), the R 2 is not calculated for 
WorkKeys. 



Table 14 



Hierarchical Lineal Model Fixed Effects, With WorkKeys as Posttest 


Predictor 


Coefficient 


df 


P 


t 


Cohen’s d 


Too 


58.547 


124 


<0.001 






Yot 


0.971 


124 


0.483 


0.703 


0.126 


Yio 


0.627 


533 


<0.001 


18.900 


1.637 



Note. y 00 = posttest score for student with pretest score at the grand mean in control group. y 01 = increase in posttest score for 
student in experimental group. y 10 = increase in posttest score for each point above the grand mean scored by a control group 
student. 
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Table 15 

Final Estimation of Variance Components for WorkKeys 




Variance component 


SD 


df 


f 


P 


Variance within 
classrooms 


a 2 = 164.877 


12.840 








Base variance between 
classrooms 


x = 17.604 


4.195 


123 


180.438 


0.001 


Full variance between 
classrooms 


x= 18.115 


4.256 


122 


180.364 


0.001 



R 2 



N/A 



Effect Size 

Overall, the results of the quantitative analyses show a pattern favoring the experimental 
students and therefore providing support for our model. But while the differences between the 
experimental and control groups may be statistically significant, are they of a magnitude to be of 
interest for policy and practice? To answer this question, we also examined the effect size, which 
is a measure of the difference between the average scores of two groups in comparison to the 
overall variation of the scores in the groups. Effect sizes (ES) can be thought of as the average 
percentile standing of the average treated (or experimental) participant relative to the average 
untreated (or control) participant. An ES of 0.0 indicates that the mean of the treated group is at 
the 50th percentile of the untreated group. According to Cohen (1988) an ES of 0.8 indicates that 
the mean of the treated group is at the 79th percentile of the untreated group (see Figure 4). 
Cohen (1988) defined effect sizes as “small, d = .2,” “medium, d = .5,” and “large, d = .8”, 
stating that “there is a certain risk in inherent in offering conventional operational definitions for 
those terms for use in power analysis in as diverse a field of inquiry as behavioral science” (p. 
25). 



In our data, the effect size on the TerraNova was 0.55, which is considered moderate or 
medium. Another way to interpret this coefficient is that the mean score of the treated group 
(experimental) is at the 71st percentile of the untreated (control) group. The effect size with 
ACCUPLACER was also medium at .42, or about the 66th percentile. The only other 
experimental study we have found regarding math learning is the Cognitive Tutor, which is 
marketed widely and has an effect size of only .22 (Schneyderman, 2001). 
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50th 

percentile 




Figure 5. Example of effect size (ES) of hypothetical control and experimental groups. 

Differences by SLMP 

To determine whether the beneficial effects of being in the experimental group differed 
within each SLMP, we ran the model for each test within each SLMP. We specifically did not 
want to compare SLMPs to each other to see if one is more significant than another; this would 
be like comparing apples to oranges since they had different CTE content and thus emphasized 
different math concepts (refer again to Appendix B). Rather, we wanted to see if and to what 
extent the intervention “worked” in each of the SLMPs. 

Because the sample size within each SLMP was not sufficient to run HLM, the following set 
of analyses used the GLM (general linear model) technique. In 11 of the 15 (3 tests in 5 sites) 
comparisons, students in experimental classrooms scored higher than control classrooms (Table 
16). Students in experimental classrooms tended to score higher on at least one of the measures 
of math achievement in each of the experimental replications (p < .10). Effect sizes ranged from 
.39 to 2.80 (but the latter was in a sample of only 4 classrooms in SLMP A, so this result should 
be interpreted with caution). 
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Table 16 

Mean Classroom Posttest Scores with Pretest as Covariate by SLMP 

Experimental Control Between-subjects effects 



Cohen’s 



SLMP 


M 


SD 


n 


M 


SD 


n 


F 


P 


d 


A 


TerraNova 


63.53 


9.15 


2 


50.82 


17.17 


4 


0.001 


0.980 




Accuplacer 


60.48 


24.92 


2 


56.38 


19.83 


4 


0.140 


0.733 




WorkKeys 


74.02 


5.04 


2 


59.60 


6.84 


4 


5.974 


0.092 


2.80 


B 


TerraNova 


43.93 


10.30 


11 


37.85 


9.76 


17 


5.524 


0.027 


0.69 


Accuplacer 


35.48 


6.70 


11 


38.47 


7.51 


17 


0.073 


0.789 




WorkKeys 


51.56 


9.98 


11 


57.44 


10.88 


16 


0.255 


0.618 




C 


TerraNova 


48.04 


11.65 


21 


46.49 


13.10 


23 


0.259 


0.613 




Accuplacer 


41.79 


9.66 


21 


37.09 


10.57 


23 


6.276 


0.016 


0.85 


WorkKeys 


61.69 


12.31 


20 


57.74 


11.94 


23 


2.728 


0.106 




E 


TerraNova 


55.81 


13.38 


9 


50.06 


13.14 


14 


3.025 


0.097 


0.64 


Accuplacer 


49.90 


7.76 


9 


49.02 


13.48 


14 


0.152 


0.701 




WorkKeys 


67.59 


12.94 


9 


65.31 


11.06 


14 


0.597 


0.449 




F 

TerraNova 


46.08 


11.51 


13 


47.06 


14.19 


16 


2.111 


0.158 




Accuplacer 


39.88 


14.31 


14 


35.82 


8.32 


16 


3.568 


0.070 


0.39 


WorkKeys 


57.11 


10.93 


13 


57.37 


13.18 


14 


1.866 


0.185 





Note. Probabilities are for one-tailed tests because of our directional hypothesis. Site D was dropped after the pilot study for 
administrative reasons. 
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The Impact of Math-in-CTE on Technical Skill/Occupational Knowledge 

Our final analysis addressed the third research question of this study: Does enhancing a CTE 
curriculum with additional mathematical instruction reduce the acquisition of technical skills and 
knowledge? As described in Chapter 3, tests appropriate for each of the five occupational areas 
were administered to the experimental and control students as part of the posttesting. One site 
usually takes a nationally-administered test that covers four different competency areas, so 
students at this site took the national tests that covered the competencies that had been covered as 
part of this project during the 2004-2005 school year. The other sites each took one occupational 
skills test. Table 17 presents the classroom means on the occupational skills tests for the five 
SLMPs controlling for pretest of math ability. The inclusion of this control was based on the 
understanding that a certain amount of math was part of these occupationally specific tests. 

These data suggest that the intervention had no negative effect on the acquisition of technical 
skills or occupational content knowledge, with the exception of Site A. Here we found that the 
experimental classes scored significantly higher than the control classes, but it should be noted 
again that because of the small n, these results should be viewed with caution. 

Table 17 

Comparisons of Experimental and Control Group Classrooms on Tests of Occupational Skills 
and Knowledge 



SLMP 


E; 

M 


tperimenta 

SD 


1 

n 


M 


Control 

SD 


n 


F 


P 


A 


67.52 


13.25 


4 


55.67 


5.44 


4 


5.685 


.064 


B 


















Test 1 


42.34 


14.19 


6 


40.39 


5.78 


9 


.147 


.708 


Test 2 


47.37 


8.05 


11 


46.12 


6.06 


14 


.195 


.664 


Test 3 


43.94 


12.84 


8 


46.33 


7.03 


12 


.357 


.558 


Test 4 


42.48 


5.28 


9 


45.57 


4.38 


10 


2.180 


.159 


C 


57.35 


9.23 


26 


56.98 


12.50 


23 


.023 


.880 


E 


49.06 


8.45 


16 


48.23 


7.31 


18 


.214 


.648 


F 


44.86 


4.68 


15 


45.44 


5.62 


18 


.090 


.766 



Note. M values are average percentage correct. In Site B. different classrooms took one of four different tests administered 
nationally in that occupational area. Adjusted means are estimated marginals with the pretest as the covariate. Probability values 
are two-tailed, as this hypothesis was actually the null hypothesis that there would be no significant differences between the 
experimental and control classrooms. Site D was dropped after the pilot study for administrative reasons. 



In the following chapter, we present the themes that emerged from the qualitative data to 
complement our quantitative findings. 



National Research Center for Career and Technical Education 



53 




Building Academic Skills in Context: Testing the Value of Enhanced Math Learning in CTE 



54 



National Research Center for Career and Technical Education 




Building Academic Skills In Context: Testing the Value of Enhanced Math Learning in CTE 



CHAPTER 6: WHAT WE LEARNED: QUALITATIVE RESULTS 

As we analyzed the multiple sources of data collected for fidelity of the treatment, we learned 
more with regard to what worked and what did not. Here, we present the qualitative themes that 
emerged from our analyses, illustrating each with what the teachers told us in their surveys and 
focus groups. 

The CTE and math teachers described their participation in this study as a positive learning 
experience for themselves and their students. They commended the Math-in-CTE model as one 
that works — a model of “true integration,” as one teacher shared: 

Ever since I have been in education, all I’ve heard is integration this, integration 
that. And I think, in this study, the math teachers and those of use in career and 
technical education realize this is true integration. None of us had seen it done. 

While the teachers indicated that participating in the study was hard work and frequently 
frustrating, they felt it was a worthwhile effort. The outcome was a model do-able for a CTE 
teacher and of observable benefit to their students. 

Many CTE teachers across the replications described how following the model strengthened 
their overall teaching skills and the delivery of their CTE content, as expressed by this teacher: 

I got into the math thing to teach the math thing, but before long I realized that it 
was helping me teach what I had to teach. . . . Whenever I taught the math, it 
helped [the students] understand the technology better. 

Participation in the study opened opportunities for teachers to share the project with their 
peers and administrators, bringing positive attention to their programs. One CTE teacher 
recounted how he taught a math-enhanced lesson with his principal observing: “I thought I might 
be sticking out my neck just a little bit. But he came and he was very excited about the program 
and it really sold the program to him. ... He thought it was really valuable stuff.” CTE and math 
teachers alike supported the model to the extent that they suggested testing it in other academic 
areas such as the sciences. 

CTE-Math Teacher Partnerships 

The CTE-math teacher partnerships that formed were highly valued by both and considered 
essential to the success of the program and of any future replications. All agreed that the 
expertise of both math and CTE teachers were needed to make this project work, as one math 
teacher noted, “The key to making this thing work is the math and CTE teacher working 
together, bringing both of their knowledge bases into this....” Many CTE teachers admitted to 
simply struggling with math and considered the partnerships critical to their ability to engage in 
the project, as one shared, “The math partners in my case were absolutely critical for me to have 
any confidence in reinforcing the elements and the concepts we were covering in the lessons.” 
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Interestingly, many of the teacher-teams reported passing through a period in which they had 
to overcome tensions and anxiety in working together, especially on the part of the CTE teachers 
who often expressed a lack of confidence in mathematics. One math teacher described his CTE 
partner’s initial experience as one of “absolute fear as he stepped into that realm he was not 
familiar with.” However, these fears dissipated and a mutual respect for each other’s expertise 
emerged, as he further explained, “We have three math teachers on the staff, and he (the CTE 
teacher) has now become a sort of de facto four. It’s pretty neat!” For the teachers who already 
knew each other and/or those who previously had worked together, the study provided a venue 
for increased collaboration. 

The math teachers began the study with the understanding that they would contribute to the 
CTE teachers’ efforts to enhance math in their courses. However, the benefit of the partnerships 
proved to be reciprocal. The math teachers described it as an “eye-opening experience,” and they 
valued learning about the CTE content and contextual learning. They also gained real-world 
applications for their own instruction, along with an increased repertoire of answers for the 
students who ask, “Why do I need to learn this?” A number of the math teachers spoke of the 
burden they experience in preparing students for high stakes math testing and expressed 
appreciation for their CTE teaching colleagues who now shared their concerns: 

One of the things I really liked was just the idea that some of the math 
responsibility fell on some other people. People wanted to take some initiative to 
help us teach the math. So often, if any score on any test is low in math, it’s the 
math department. . .No one ever thinks that other areas could help as much as they 
really can. . . . that whole idea of sharing the responsibility. 

CTE teachers from all SLMPs noted how essential the math teachers were to the success and 
credibility of the project in their schools. However, in a turn of the table, one math teacher noted 
how the program lent credibility back to the math teachers: 

I think this built the credibility of the math teacher, both for the kids and for the 
technical teachers. Now I think a lot of times in the school. . .the math teachers are 
looked at like the slave drivers who are just business-like all the time. And that 
broke that down a bit for both the technical teachers and for the kids, because the 
kids saw the technical teacher looking to me for assistance. . .and they were kind 
of like, “Hey, he thinks that she’s OK. And if he thinks that she can help him out, 
then maybe she can help me out, too.” And at the same time, with the technical 
teacher I became someone that wasn’t that stick in the mud slave driver, boring 
math teacher. I became a person. 

By the end of the full-year implementation, both groups of teachers reported less need to 
meet formally due to the CTE teachers’ increased confidence with the math and familiarity with 
the lessons. However, CTE teachers still expressed their need for immediate and nearby support 
of the math teacher, especially to check the accuracy of their lessons or to get help in bridging 
the math to the CTE. Many of the teams indicated that they intend to keep their collaboration 
going after the end of the study. Some have made presentations at conferences and others have 
been called upon by their administrators to lead curriculum integration efforts in their schools. 
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More Than a Set of Lesson Plans 

The treatment in this experimental study was unique in that it emerged as a truly 
collaborative effort between the researchers and the teachers. Together, Center researchers and 
SLMP facilitators created a framework and facilitated a process that enabled CTE-math teacher- 
teams to generate and implement the treatment. This level of engagement in the process created 
enthusiasm among the teachers, as one commented, “Just the fact that we were the ones heading 
this up I think it made it kind of exciting to see what might happen with some of the things we 
had developed,” Teachers reported that the actual structure of the professional development and 
pedagogic framework made it possible for them to work together effectively, as one observed, “It 
was structured, [but] we were given a chance to do it. We weren’t just kind of thrown together, 
we were given tasks.” 

As the study progressed, the teacher-teams and SLMP facilitators began to experience 
something more than just workshops and lesson plans. One teacher observed that “the whole had 
become greater than its parts.” Others described how they looked forward to the collegiality and 
coming back together to share their ideas with and present their lessons to the whole group. One 
jokingly described it as returning to the “mother-ship.” The math teachers increasingly found 
value in sharing their math ideas with one another and the CTE teachers, as one math teacher 
explained, “The expertise in these rooms for these last 2 years [sic] was just invaluable. And I 
think as teachers it doesn’t matter whether it was math or CTE. We each learned from everybody 
else.” A CTE teacher echoed the same thought: 

What was really interesting was when we began the professional development 
meetings, there were different math partners, and we had all these minds thinking 
about these lessons.... The mixing of ideas as you’re teaching the lesson is 
amazing — it’s that diversity — it’s coming back here every so many months and 
knowing how everyone else is doing. 

Another CTE teacher described the experience this way: “It wasn’t just me and one other 
person. It was the group. . .and we were up there putting a lesson together and you just get into 
it — ‘hey, try this’ — ‘hey, I tried that. . . This process of teacher-teams joining in with other 
teacher-teams, developing collegial relationships while working with the lessons and math 
problems was not something that we planned or anticipated when the study was initiated. 
Nevertheless, these emerging communities of teacher-learners were a welcome outcome with 
significant impact. 

At the end of the study, teachers and researchers concurred that the Math-in-CTE model 
could not be reduced simply to a set of workshops and lesson plans, but that indeed it was the 
process that reaped the results. In fact, teachers across the study were reluctant to recommend 
packaging the lessons for teachers who had not experienced the process, as reflected in this 
comment: 

You can’t just walk into those school districts and say ‘here it is’ ... . They’re 
going to have to develop it themselves... This is not an overnight thing.... you’re 
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not going to be able to plop a book down in front of [the teachers] and say ’here it 
is.’ 

Teachers emphasized the need for those interested in implementing the model to be engaged 
in the “whole process, from day 1” with a math-teacher partner and the full complement of 
professional development activities including concept mapping. As this teacher put it, “the 
lessons are just what came out of all this work.” 

The “Tipping Point” 

At the heart of this study was the principle that the enhanced lessons should emerge from the 
math embedded in existing CTE curricula. They were not to become “add-ons” or special stand- 
alone lessons. The CTE teachers widely favored and supported the model, however, they 
continued in their struggle to implement the lessons while covering their established curriculum 
and fulfilling other responsibilities. We know from the reporting data that not all the CTE 
teachers in the study were able to teach all of the lessons as scheduled all of the time. This 
finding was also confirmed in the focus groups and surveys. When teachers were asked for 
reasons why they did not teach all the lessons, most indicated they ran out of time in their term or 
school year to teach the lessons and/or that they gave precedence to other instructional-related 
activities, for example, keeping up with required curriculum, assuring certification requirements, 
meeting mandated standards, preparing for student competitions, etc. Interferences typical to 
schools, such as snow days, athletic events, and graduation events were also noted. 

We began to identify this tension as the tipping point because the teachers commonly 
expressed the experience in oppositional terms. For instance, a teacher would say, “If I do a 
math-enhanced lesson, then I will lose time in the shop,” or “I had a snow day, therefore I did not 
have time left to teach the lesson.” In this sense, the tendency to view the lessons as “add-ons” to 
the curriculum was not fully resolved for these teachers in the full-year implementation. It is 
worth noting, however, that a number of teachers refuted this dichotomy by removing “fluff’ 
from their curriculum and becoming more efficient in using time in their classrooms, 
laboratories, and shops. They perceived the value of the math-enhanced lessons to be greater 
than what they replaced, as one explained, “Some fluff I was able to get rid of, because I just 
thought the math was more valuable to them at this point.” Others claimed that the math- 
enhanced lessons were worth the trade-off by bringing “more relevance and importance to shop 
time” or by helping students see “what they needed to make a hospital run better.” It was a 
balancing act that one teacher summed up like this: 

There’s a finite amount of time — you don’t have all the time in the world. I cut 
back on some of the just plain old textbook lessons (the “do the questions in the 
back of the textbook” assignments)... there was a trade-off. That’s the only way 
that I found that I could make it work. . . 

Some of this balancing act was a function of the research study itself. The fit of the lessons to 
the curriculum, identified in the pilot study as a barrier to implementation, improved 
considerably in the full-year study with the implementation of the scope and sequence planning 
charts. And, as we reported earlier in this report, the teachers were given the freedom to schedule 
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the lessons where they best fit in their own curriculum. However, the teachers were still required 
to implement the full set of lessons agreed upon in their SLMP. This caused some problematic 
issues when the common, agreed-upon lessons were not an ideal fit to each individual teacher’s 
curriculum. Recognizing the importance of consistency in the research, many teachers reported 
adjusting their curriculum to accommodate the lessons, and in turn, “tweaking” the lessons to 
accommodate their curriculum, as expressed by this teacher: 

The. . .lessons that we had for the study were not all appropriate for what we 
taught in the curriculum this year. . . . There were times when we had to plug in, I 
won’t say inappropriately, but material that we weren’t actually covering in the 
normal course of study. . . . You knew it going in and you made it fit. 

The teachers predicted this condition would be remedied in the future when they have full 
liberty outside the bounds of the study to schedule the lessons independently. 

“Awesome” Lessons 

The lessons that the teacher-teams produced for the study were a point of ownership and 
pride for the teacher-teams. Overall, the CTE teachers enjoyed developing and teaching them, as 
one claimed, “Our lessons are absolutely awesome.” A majority of the teachers reported they 
will be using the lessons in the future, but selecting those they use more carefully, customizing 
others for a better fit, and even writing new ones. The math teachers were also anxious for the 
study to be completed so they could implement the examples and ideas into their own 
classrooms. 

The CTE and math teachers across the study generally accepted the seven elements as a 
framework for developing and teaching the lessons. The pedagogic framework provided a 
structure that helped some teach more effectively, as expressed by this individual: 

I think it helped, basically it helped me learn how to teach better. ... I guess they 
try and teach you those kinds of things in education classes in college and 
everything. But this kind of put it to a real world situation here, where I had every 
time I taught these; I was using those same elements. I guess it gave me a better 
sense of teaching. 

One math teacher noticed that the organized structure and flow of the seven elements was 
particularly well-received by their special education students. A few teachers and their math 
partners, who agreed with the framework in principle, felt that some of the lessons became too 
long and/or too redundant in some of the steps, especially for those students who already 
understood the math. They reported adjusting certain aspects of the elements in the lessons to 
more closely accommodate the abilities of their students. Those who found the framework a little 
too restrictive reported varying the sequence of some elements to make the lessons more 
engaging for their students. 

The CTE teachers found that progressing into the traditional math problems in Element Five 
of the lesson plan was the most challenging and problematic for themselves and their students. It 
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was at this point in the lesson where the CTE teachers presented the math without its context and 
often struggled to keep their students’ interest as they recognized the lesson as math. However, 
this was also the point at which the bridging was complete and “the light bulbs went on” for 
many students. We found that Element Two, which involved the assessing students’ math 
awareness at the beginning of the lesson, was the least understood and practiced element of the 
framework. Both Elements Two and Five evoked more anxiety for the CTE teachers who felt 
that this might not be in their realm of expertise or responsibility. 

Interestingly, the math teachers noted that the movement from the embedded example to the 
traditional example framework was an effective counterpoint to their own approach, which 
typically began with the purely academic concepts out of any context. As one math teacher 
noted, “. . .you had something to relate it to. . .you didn’t have a dry, boring ending. You were 
able to apply it to something and make it a little bit spicier.” The math teachers expressed having 
limited time to spend on contextual examples to reinforce the many math concepts they are 
required to teach. Therefore, they considered it highly complementary to their instruction to have 
the CTE teacher start with a contextual example and bridge it back to the traditional examples 
they had already introduced in their math classes. This led some to recommend more future 
collaboration with the CTE teachers to maximize their opportunities to address the same 
concepts. 

The CTE and math teachers both commented on how much math they found in the CTE 
curriculum, noting that there is much more to do. This was just the beginning, as many 
expressed, “There are so many possibilities for added curriculum!” One teacher described how 
one lesson could spin off several more in an on-going process of development. In one focus 
group session, math and CTE teachers discussed the possibilities of CTE courses being offered 
as for-credit applied math courses. While CTE teachers warned against losing the integrity of the 
CTE content of their programs with too much math, they agreed that the students were benefiting 
from the experience and for that reason alone, they should not give up on the effort to enhance 
the math in their CTE courses. 

“Getting It” — The Student Experience 

Both CTE and math teachers across the study shared many anecdotes of students who 
otherwise did not excel in math but were now “getting it” and even enjoying the lessons. One 
CTE teacher related her success story: 

One day [a student] remarked that she was flunking math. I asked her why and 
what she was struggling with, because she did so well in the math-in-CTE 
assignments. She said they were “different.” I began helping her with her math 
assignments and related them back to what we had done in CTE. She started 
seeing the connections and now has an “A” in her math class. 

The CTE teachers frequently reported initial resistance from students followed by break- 
throughs in their learning, as this teacher described: 
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Some of them moaned. But then later on they came to us and said, ‘Oh, thank 
you. We understand it now’ . . .it made a huge difference to a lot of kids. Some of 
the kids in the beginning of the year said, ‘I can’t do this, I can’t do this.’ At the 
end of the year they said, ‘I get it! That was great. I really liked that!’ 

The teachers in focus groups agreed that lessons did work best when introduced as CTE 
lessons and were not pronounced as anything different. However, they also acknowledged that 
students were quite aware of the math-enhanced lessons, if not told directly each time, they 
figured it out. Because this foreknowledge was a function of the study, CTE teachers anticipate 
that when they teach the lessons in the future their students will likely not know the difference. 

Challenges 

While the success stories were widely shared, there were teachers who openly expressed their 
doubts about the outcome of the study and its benefit to students. Their concerns were primarily 
wrapped around the notion that CTE teachers are not math teachers, they really aren’t trained to 
teach math, and that this kind of effort in a classroom takes away from the CTE they should be 
teaching. Some teachers reported that, by the end of the year, students were getting tired of the 
lessons, and related this reaction back to the length and number of lessons and the need for 
systematic implementation throughout the year, as noted here: “You have to spread those lessons 
out. You can’t try to cram, because the kids. . .get a little tired of it if you try to stick too many of 
them together.” Others expressed concerns that too much math in their curriculum could 
potentially impact the enrollments of their classes, emphasizing that students come to class 
because they enjoy the CTE and do not expect to learn math. 

The CTE teachers continued to express their surprise at their students’ lack of readiness in 
math. As lessons were being developed, NRCCTE researchers and SLMP facilitators stressed the 
importance of developing math-enhanced lessons at a level of Algebra I or higher. However, in 
practice, the teachers found out that all too many students do not possess the most foundational 
math skills such as measuring, multiplying, simple ratios/proportions, etc. There were reports 
from teachers who were not able to teach even the most basic algebraic concepts without 
remediation. This often made the lessons longer than planned and took precious time away from 
the CTE content, as one explained, “I know they [the researchers] wanted a higher order of math, 
and sometimes it took a half-hour to bring your kids up to that level. So besides just teaching the 
lesson, you had to get your kids up to that level.” This also gave rise to questions about their role 
in teaching the math. In one focus group, teachers openly debated how much responsibility CTE 
teachers should assume for assessing the students and then catching them up on the math they 
should have learned before entering the upper grades. After all, they are not math teachers. 

Should it be their responsibility to take the time to do this? This was one teacher’s response to 
that argument: “We can impact math for these kids. We can. This project proved that to me.... It 
is part of our curriculum to have them leave us with these math concepts. .. .” 

Teaching to the mixed math abilities of the students also continued to be a concern and a 
frustration for the CTE teachers. While lesson plans and materials developed in the pilot study 
were significantly improved for this purpose, challenges remained. Some teachers expressed 
concerns about meeting the needs of their advanced students who found the lessons redundant 
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and boring. However, others reported that their advanced students were gaining benefit by seeing 
the application in the workplace; while they may have excelled at the math in a traditional 
classroom setting, they could now see how it was put to a practical use. This, of course, was a 
strong motivator for students of all abilities, as one teacher commented, “The lessons enabled 
[students] to understand more math concepts, to see how math relates to the ‘real’ world and it 
boosted their confidence in their math ability.” 

The teachers seemed to agree that most students benefited and that the lessons were worth 
teaching even if the higher level students knew the math. They described situations like this 
where students spontaneously helped one another: “I think we should bring out... the amount of 
the cooperative learning that went on. One kid says, ‘I don’t get this,’ and the next thing you 
know three kids are over there. . .they really did pull each other along.” A number of teachers 
reported the advantages to all students when they grouped students of mixed abilities and 
allowed them to engage in peer- teaching. One teacher in the study, who had only special 
education students in his program, offered this observation, “My special education students 
walked away from these [lessons] with pride that they know more than what is expected in their 
regular classroom.” Interestingly, the CTE-math teacher-partners in smaller schools, who taught 
the same students, noted examples of students who exhibited increased confidence and interest in 
math because they observed the math and CTE teachers working together. 

If the Teachers Were Consultants... 

At the end of the study, we asked the CTE and math teachers what they thought it would take 
to successfully replicate the model in the future. There was consensus across the SLMPs that it 
must begin with teachers who are willing and teachable. The upshot is that the model is a process 
and not just a package of lessons, and it is a model that requires engagement of the teacher. The 
model works, but it is hard work, and it won’t work unless the teachers want to participate. 

The teachers provided a mix of thoughts about the extent of the professional development 
needed for future participants, however they agreed that all would need dedicated time away 
from the demands of their schools in order to work together on the project. They also expressed 
that teachers, on the whole, feel overworked and are reluctant to leave their classrooms, so the 
effort must be presented as something worth their time, both in fair compensation and in 
knowing that the effort will result in lasting and positive results for their students. As one teacher 
pointed out, “The buy-in will be in the results.” The teachers also discouraged the notion of 
putting together groups of teachers from a broad range of content areas indicating that “what 
worked” was a critical mass of CTE and math team-teachers working together within the same 
SLMP. 

Overall, the math cluster meetings were perceived by the CTE teachers as ‘fun,’ but 
unnecessary to the process. That is, the food was good and the socialization component was nice, 
but on the whole it was of less use for math support. What the CTE teachers valued most was the 
one-on-one math support provided by their math-teacher partners in their individualized 
preteaching meetings. In the end, these partnerships were deemed to be critical to the entire 
process, and as one math teacher joked, “You just have to have a ‘good marriage’ between the 
[CTE] teacher and the math teacher.” Teachers also pointed out that the partnerships worked best 
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when the math teacher was in close proximity to the CTE teacher, preferably in the same 
building with joint planning periods. 

Interestingly, the teachers acknowledged the necessity of someone to watch over their 
progress. The SLMP facilitators and Center staff monitored the teachers’ scope and sequence 
charts and making regular requests for their reports and other data. This “nagging factor,” as it 
was fondly named, was an essential aspect of accountability in keeping teachers on track with the 
lesson implementation. However, they also noted that while the scope and sequence was 
necessary as a broader organizer in the sense of fit to the curriculum, it was less than desirable as 
a calendar schedule. Many felt it was too rigid, and that more flexibility would be desirable in 
the future implementations. 

The CTE teachers were quick to point out that state standards and high stakes testing count 
for something at their schools. They were anxious to share the findings of the study with their 
peers and administrators, thus demonstrating their contributions of CTE programs to the 
academic achievement of their students. This pressure to justify CTE programs through 
standards and test scores tempted some to suggest that the math concepts (as opposed to the CTE 
concepts) might be the starting-point in the curriculum mapping process. This tension revealed 
the fine, but important, distinction between the NRCCTE model of enhancing math that occurs 
naturally in CTE from other models that overlay math onto the CTE curriculum. 

Most of all, both the CTE and the math teachers expressed a desire to see this effort continue. 
Having seen the benefits of the study for students and teachers alike, a math teacher made this 
reflection: 

I thoroughly enjoyed being a part of this process, and I feel very privileged that I 
was selected by my director and my CTE teacher to be a part of this. I made great 
friendships here. I’ve learned a lot, and I still have a lot to learn. I just think it’s a 
wonderful thing. There’s nobody that can convince me that this is not the way to 
go, because of seeing the results in my small little school.... 

Finally, we close with this final thought from a CTE teacher: 

I have been in education for a few years and I’ve been involved in a lot of 
initiatives. This is the best one I’ve ever been involved in and the outcome has 
just been so positive. It’s a great thing we did. But I get disillusioned with ‘this 
year’s new thing and next year’s new thing,’ you know what I mean? ... It falls 
right through a hole. This is too good of a thing to let it just fall through a hole. 
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CHAPTER 7: CONCLUSIONS AND IMPLICATIONS 

In this chapter, we present our answers to the research questions that guided this study and 
discuss the implications of these answers for policy and practice. The study emerged from a 
belief that has been discussed in the literature but that has rarely been rigorously tested: Students 
enrolled in high school CTE courses, who are more explicitly taught mathematics concepts 
embedded in the curriculum, will develop a deeper and more sustained understanding of 
mathematical concepts than those students who participate in the traditional CTE curriculum. 

In other words, the primary research question of this study was whether a math-enhanced 
CTE curriculum would improve math performance among high school CTE students as 
measured by traditional and applied tests of math skill. To test this question, we designed and 
conducted a group randomized experiment (Murray, 1998) that involved 131 teachers and almost 
3,000 of their students from five different occupational areas: agricultural mechanics, auto 
technology, business and marketing, health, and information technology. Random assignment of 
teachers to experimental and control conditions yielded groups of students with equivalent 
performance on a math pretest. Therefore, the fact that the experimental classrooms showed 
higher posttest scores than the control classrooms on all three tests, albeit in different 
occupational areas, provides evidence that the experimental intervention was successful. In 
addition, there were no significant group differences on the tests of technical skills and 
knowledge in students’ occupational areas, demonstrating that mathematics can be enhanced in 
CTE without detracting from content learning. 

Finally, increased scores on the ACCUPLACER test also potentially demonstrate the need 
for less remediation at the postsecondary level. 12 However, there is no set cut score on the 
ACCUPLACER indicating the need for remedial work, and there has been no real data or 
research on how to use college placement tests to determine the degree to which students have 
mastered the content experts say is necessary to be ready for college (Venezia, Kirst, & Antonio, 
2003). Colleges use placement tests to decide which course students should take, including 
whether or not they need developmental (remedial) courses. Presumably, these placement tests 
are tests of college readiness. For example, ACT has COMPASS and SAT has ACCUPLACER. 
There are three problems with using placement tests as a criterion for college readiness. One is 
that there is no universal cut-point. For both the ACT and the SAT tests, setting cut scores is a 
local option. Second, in addition to the ACT and SAT tests, many universities and higher 
education systems have their own placement tests. Thus, depending on where a student goes to 
college, he or she may take any one of nearly a hundred different placement tests being used in 
the United States at this time (Venezia, Kirst, & Antonio, 2003). Third, the validity of the 
placement tests and their cut scores are unknown. 

One question raised by our overall findings is how to explain the different results provided 
by the different tests. Recall that we discussed in Chapter 3 why each test was selected. We 
selected these tests because we anticipated that each might yield a different result. As discussed 



12 

ACCUPLACER is widely used to determine if entering college students need noncredit, developmental courses. 
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in Chapter 3, the TerraNova was used as our traditional measure of math achievement, the 
WorkKeys was presumed to reflect an applied approach to measuring math achievement and the 
ACCUPLACER was included to assess the potential impact on remediation of this approach. 

Because each test maker employs its own conceptual framework for math, we assumed each 
approached the construction of math questions around similar concepts in different ways. We 
had anticipated that our approach would yield the greatest effect with the WorkKeys measures 
and were surprised at its low sensitivity to our interventions. The WorkKeys test provided its 
takers with the additional challenge of language. Students needed to read through the scenario, 
pick out the important information, and solve the problem before them. In the case of students 
with weaker English skills, the WorkKeys test may have been measuring more than applied 
mathematical ability. The test may have also been measuring student’s abilities to read and 
successfully decode the English language. 

Nonetheless, this study was not designed to test the tests or explain what we had anticipated 
to be different results, rather we chose to use existing tests with established reliability and 
validity to ensure an unbiased measure of the effect of our treatment across various replications. 
Future analyses of these data may provide useful information to the test vendors regarding the 
varying sensitivity to the Math-in-CTE intervention. 

It is also useful to place our findings in the context of the high school experience. Our study 
tested the effect of using a carefully crafted pedagogic and professional development approach in 
a single class. These results, while very encouraging, were not expected to resolve or necessarily 
remediate all of the well-documented shortcomings in adolescents’ mathematics ability. We 
made no assertions that this one modest intervention would result in students passing math exit 
exams or eliminating the need for remediation. We do suggest that it is a way, but not the way, to 
help United States youth gain greater mastery of the math critical to their workplace success and 
post high school education. 



Threats to Validity 

In the Campbell and Stanley’s (1963) classification of research designs, ours was Design 4, a 
pretest-posttest control group design. This design controls for seven of the “different classes of 
extraneous variables [that] might produce effects confounded with the effect of the experimental 
stimulus” (Campbell and Stanley 1963, p. 5). Thus factors such as the history, maturation, 
differential selection, and instrumentation were controlled for both groups. Since the 
experimental and control students both took the pretest, any sensitization to mathematics was 
also constant for both groups. Indeed, if control teachers became sensitized to mathematics in 
their curriculum and acted on this, we would be underestimating the effect of the math-in-CTE 
approach. 

The fidelity of treatment measures we used revealed variation across the SLMPs on several 
dimensions, but our attempts to relate measures of these dimensions to posttest performance 
were for the most part inconclusive. We did not find number of lessons, length of lessons, or 
ratings of their delivery to have any consistent relationship with students’ outcomes. 
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One factor that randomization does not control is attrition. If there is differential attrition 
from the experimental and control groups, this, and not the experimental intervention, may be the 
cause of any differences found between the groups. The key word, of course, is differential. For 
example, if the experimental teachers who withdrew tended to have lower performing students 
and the control teachers who withdrew tended to have higher performing students, this could 
result in the experimental group scoring significantly higher on posttests regardless of the effect 
of the intervention. 

We do not consider attrition a significant threat to the validity of our findings. The attrition 
that occurred was primarily between the pilot and full-year study; there was none during the full- 
year study itself. Most of the withdrawals among experimental teachers were for reasons such as 
changes in their teaching or family responsibilities, geographic moves, and in a few cases, 
sickness. Most of the control teachers elected to continue, but because of the attrition in the 
experimental group, the number in the control group was reduced on a random basis to make the 
two groups more equal in number. Comparisons of the pretest performance of students of the 
continuing and noncontinuing teachers showed that they did not differ significantly. 

The teachers who volunteered to participate likely shared more measurable and 
unmeasurable attributes than teachers who did not, which might include a higher level of comfort 
in teaching mathematics 13 . We acknowledge other possible differences as a limitation to the 
generalization of our findings but not to the validity of the group comparisons, because the 
random assignment process distributed self- selection attributes to both groups equally. 

While our teachers are atypical, this does not mean that their students are. In high schools 
and career centers, students do not choose their teachers. They choose CTE courses because of 
an interest in an occupational area, not because of who teaches the classes. Teachers who have a 
good reputation may attract students to their areas, but because of randomization any such 
selectivity in students would be distributed across the experimental and control groups. 

With regard to the external validity, or generalizability, of our sample, we make no claim that 
the CTE teachers who participate in this research, both as experimental and control, are 
representative of all their colleagues. The fact that they volunteered to participate in the research 
and then continued from the pilot study to the full-year study defines them as atypical. As 
suggested earlier, most would probably be classified as early adopters, teachers who continually 
seek ways to improve their professional skills and knowledge. 

All we claim is that when a self-selected group of CTE teachers taught math-enhanced 
lessons that they had developed, their students performed better on standardized measures of 
math achievement than the students of similarly self- selected CTE teachers who did not teach 
these lessons. Whether nonvolunteer teachers would be willing to become involved to the extent 
that ours did, is a researchable question. 



13 There was no significant difference between control and experimental groups of CTE teachers on anxiety about 
teaching math at the start of our study; however, we did not have a nonvolunteer comparison group against which to 
compare our CTE teachers. 
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At best, our intervention was a small part of the students’ total exposure to instruction. If one 
assumes 180 school days, and six 1-hour classes per day, the total number of instruction hours in 
an academic year are 1,080. The reports submitted by the CTE teachers indicated that across all 
five SLMPs about 20 hours were spent teaching math-enhanced lessons. This represents 11% of 
1 class period during the academic year, or less than 2% of total instructional hours in a school 
year. Despite this modest time investment, there was still a significant, meaningful effect on 
math achievement. Mathematical problem solving, which requires students to apply knowledge, 
skills, and strategies to novel problems, is a very difficult form of transfer to affect (Bransford & 
Schwartz, 1999). Our model, however, appears to have helped students transfer their math 
concept learning from their CTE courses to global tests of mathematics performance. The rigor 
of the research design, the number of participants and the breadth of the occupational areas 
constitute a rationale for generalizing the findings from this study to most CTE courses that are 
classified as providing Specific Labor Market Preparation (SLMP). 

Schlechty (1997) reminds us that the purpose of schooling is to engage students in 
intellectual work that results in usable skills rather than focusing on testing. We believe that both 
need to be accounted for in schools. We do not mean to emphasize testing, but we believe that 
because it is likely to continue, and likely to have broad-ranging effects on individual students, 
that it is important to include it among the various contexts for which students should be 
prepared. 

In this study, we concentrated on a single CTE course. If explicit mathematical enhancement 
were practiced whenever the opportunity arises in an occupational curriculum, for example, if we 
concentrated on a CTE program where students enroll in 3 or more related CTE courses (Plank, 
2001), it seems almost certain that the magnitude of the effect would be greater. 

Our overall conclusion is that enhancing instruction in the mathematics that is inherent in 
the curricula of five diverse occupational areas improves the performance of students on 
standardized measures of mathematical achievement, and it does so without negatively affecting 
the acquisition of occupational skills and knowledge. In the following section, we will discuss 
the model as it related to whole school reform and present the emergent core principles 
underlying the successful implementation of the NRCCTE Math-in-CTE model. 

Implications 

The Math-in-CTE Model and School Reform 

We recognize that the differences in the posttest scores of the experimental and control 
groups were not large, but neither was the amount of instructional time devoted to enhancing 
mathematics. We think if the approach we developed were to become general practice in CTE 
courses, students’ command of mathematics would increase substantially without negatively 
affecting the learning of occupational skills. What would be required for ongoing mathematical 
enhancement to become the norm? 

When we began this study, our focus was on improving mathematics instruction in CTE 
courses. If the words “school reform” were mentioned in our planning sessions, it was just a 
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passing reference. We did not recruit schools to take part in the study, nor did we have any 
intention or strategy for changing schools. We did, however, achieve our objective of improving 
students’ performance on standardized measures of mathematics, and this is the most important 
criterion by which school reform is measured (Borman et al., 2005; Leithwood, Louis, Anderson, 
& Wahlstrom, 2004). As we progressed through the year, we collected various kinds of data that 
helped us leam more about the treatment/intervention in this study. The teachers who 
participated in the focus groups at the conclusion of the study were particularly helpful in 
identifying what worked and what did not. From analysis of multiple sources, we have seen the 
following five core principles emerge: 

1. Develop and sustain a community of practice among the teachers. 

2. Begin with the CTE curriculum and not the math curriculum. 

3. Understand that math is an essential workplace skill. 

4. Maximize the math in the CTE curriculum. 

5. Recognize that CTE teachers are teachers of math-in-CTE, and not math teachers. 

Each will be further explained below. 

Community of Practice 

The necessary condition for successful replication of the NRCCTE Math-in-CTE model is a 
group of CTE teachers from a single occupational foci and their math-teacher partners working 
together in a community of practice to identify the math inherent in unique occupational 
curricula (e.g. auto technology, health, IT). Our teachers worked together to develop lessons to 
enhance instruction in the math. Much of the time spent in professional development with the 
experimental teachers in this study involved writing, critiquing, and revising the math-enhanced 
lessons, but the lessons themselves were not the key outcome. Rather, it was the creation 
process, the development of these lessons, that served to increase the CTE teachers’ 
understanding of the mathematics they were going to teach. It is trite, but nevertheless true, to 
say that teachers cannot teach what they do not know. Before working on these lessons few, if 
any, of the experimental CTE teachers understood concepts such as order of operations, 
measures of central tendency, and the use of proportions to solve for unknowns, well enough to 
explain them to their students. The professional development sessions provided both a structure 
and a supportive setting in which the CTE instructors could leam the math they would be 
teaching. 

A single CTE teacher working with a math colleague will be more effective than either of 
them working alone; but if they can interact with several others who are focused on the same 
objective, the effect will be exponential. This is why communities of practice are critical to 
replication success. Important to the creation of communities of practice is the issue of critical 
mass. There is a minimum number of CTE-math teacher-teams necessary to make this process 
work. While we have no empirical base to justify a number, conversations with teachers and our 
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own observations lead us to believe that at least 10 CTE teachers, each with a math partner, is a 
minimum size needed to build the kind of relationships we observed. 

Each of the experimental teachers who participated in this study taught their CTE classes 
somewhat differently. Lengthy discussion, flexibility, and compromise were required to find 
common mathematical concepts that all teachers within a given SLMP could teach. But the 
teachers in this study were able to do so, and the dialog necessary to reach consensus, as well as 
group support and critique, produced a sense of ownership in the final set of lessons that 
emerged. 

The feedback in the focus groups indicated that this process of lesson development 
contributed to the sense of shared commitment and enabled the sharing of ideas, which are the 
defining characteristics of communities of practice. Participants in the focus groups also noted 
that when mathematics and CTE teachers work together in this way, both benefit. The 
mathematics teachers found real world examples to enrich their classes, and the CTE teachers 
developed a fuller understanding of the mathematics inherent in the occupational skills that they 
teach. 

We wish to stress that this approach is not team-teaching, nor is it just a set of well-packaged 
lesson plans. The CTE teachers built their lessons in teams and refined them within their group, 
but they taught them on their own. The math teachers assisted in the development of the lessons, 
clarified the concepts, and provided teaching suggestions, but the CTE teachers delivered the 
lessons. They would not have had the expertise, confidence, or buy-in had they not spent the 
time in this process. Obviously if the model is to be adopted, time must be provided for math and 
CTE instructors to engage in such a process. Providing the structure for reform is critical, as 
many experts on change emphasize (e.g., Newman & Wehlage, 1995). Teachers need the time 
and space to create a supportive community for professional development. 

Begin with the CTE Curriculum 

This study tested one of the primary claims of CTE: that relevance facilitates learning. 
However, in doing so, we also considered it essential to maintain the integrity of the CTE 
curriculum. Since its inception as a part of the high school curriculum, CTE has been linked to 
labor market needs. These links to the workplace are what attract CTE students and provide the 
engagement that they often find lacking in academic courses. For these reasons, we required that 
the math to be taught as part of the CTE courses should emerge from the curriculum — not be 
superimposed into it. 

We acknowledge that there are many ways to think about integrating academics into 
occupational curricula. Many of these, we believe, are context-based not contextual. When the 
primary driver for curriculum reform is the academic content, whether driven by state standards 
or course content (e.g., Algebra), in a context-based approach the learning may begin to resemble 
a traditional math class except with examples presumably more connected to the students’ 
interest. In the early stages of this study, we found many such examples, e.g. “Algebra for 
Carpenters” and the like. Yet the examples used are still abstracted from the context, rather than 
flowing from a real problem the student is encountering in the shop, lab or CTE classroom. 
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One of the most difficult aspects of implementing the Math-in-CTE model was identifying 
the common math concepts in the curriculum taught by the teachers within each of the 
occupational areas. We did not attempt to identify concepts that were common across 
occupational areas, but we did ask the teachers within each area to agree on concepts that they all 
could teach. They struggled to do so, but after many hours of discussion, each area developed a 
list of concepts that served as the starting-point for the development of lessons. We began to 
think of this process as “interrogating the curriculum.” One unexpected artifact of this process 
was that the CTE teachers developed a deeper understanding of their own curriculum as they 
shared teaching strategies with each other. Notably, the research staff in each SLMP did not 
assume the role of content expert. The teachers were considered the experts, and the 
responsibility was placed on the CTE-math teacher-teams to identify the math that already 
existed in the CTE content. 

In all cases math was present in the curricula, but was expressed in terms of specific 
occupational applications rather than as math principles. In health occupations, for example, 
students learn to use goniometers to measure range of motion. A goniometer is essentially a 
protractor with two moveable extensions. The protractor is placed at the knee or elbow and the 
extensions are aligned with the upper and lower leg or arm to measure the degree to which the 
limb can bend. This specific occupational application became the starting-point for a lesson on 
the geometry of angles. Similarly, a lesson in estimating the extent of bums to a patient’s arm 
and torso became the basis for a lesson on calculating the surface area of cylinders, spheres, and 
rectangles. 

Guided by the seven elements, each of the lessons developed for this study started with a 
specific occupational application and expanded to present the general mathematical concept 
inherent in that application. While the CTE teachers addressed math in depth, making its 
applications in CTE more explicit, they were still teaching CTE, not math lessons. 

Understand Math as an Essential Workplace Skill 

The most consistent message of the past two decades of educational reform is that high 
school students have not acquired the literacy and mathematical skills required for the United 
States to remain competitive in the world economy, or at a personal level, to qualify for jobs that 
pay enough to support a family. In an age of instantaneous communication, a nation’s most 
valuable resource is the ability of its workers to access and use information. Mathematics in such 
diverse applications as statistical quality control, computer spreadsheets, and precision farming 
has become a basic component of many jobs. Technological trends imply that mathematics will 
become increasingly pervasive in most occupations that require specialized preparation. 

CTE courses have always included mathematics, but their instructors, who are not 
mathematics educators, often use “tricks of the trade” without being explicit in addressing the 
math essential to the task. For example, students learn the 3-4-5 rule to measure a square comer, 
but the source of this rule, the Pythagorean theorem, is usually not mentioned. Such an approach 
addresses the immediate task but does not assist students to generalize beyond the specific 
application. The contextual approach that we tested with our Math-in-CTE model moves 
students successively from the specific to the general. 
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A mindset we sought to establish with CTE teachers who participated in the study is that 
math is a necessary tool in the workplace. Like any other tool, it has its place in the toolbox 
required to solve genuine workplace problems. The mechanic may reach for a wrench or a 
formula to determine how to improve the performance of an automobile. The marketing CTE 
teacher will teach advertising, marketing research, statistics, economics, and the like. For all 
CTE teachers, math is part of their curriculum and it is part of the workplace, and they should 
share that reality with their students. Students will need to understand mathematics in greater 
depth if they are to be prepared for the accelerating rate of change in jobs. 

Maximize the Math 

Understanding the CTE curriculum to be rich with math, our fourth core principle is to 
encourage the CTE teachers to maximize the math whenever the opportunity arises in the 
curriculum. We reasoned that the emerging communities of practice would increase the CTE 
teachers’ understanding and comfort in teaching math. Instructors in each SLMP developed a set 
of lessons, which they agreed to teach. We encouraged them to go beyond these specific lessons 
and reinforce the concepts presented in them whenever they were teaching content that touched 
upon the underlying math. 

Another aspect of maximizing the math included constant and consistent bridging of the 
math and CTE vocabularies. The CTE teachers themselves identified the importance of moving 
back and forth from CTE to the math terminology in helping students make the link (modeling 
transfer). The teacher- teams were also encouraged to develop more instructional materials that 
met more levels of student math abilities. We did not establish methods to monitor the extent to 
which they did this, but discussions in professional development sessions and comments in focus 
groups suggest that some teachers practiced ongoing reinforcement. 

Teachers of Math-in-CTE 

The process we recommend does not attempt to make CTE teachers into math teachers, but it 
does yield expanded and improved instruction of mathematics in CTE courses. In our model, the 
role of the math teacher is to serve as a resource, a source of information and support. 
Throughout the professional development provided the experimental teachers, we stressed the 
partnership between the CTE and math instructors. We made an explicit decision not to refer to 
the math teacher as a coach or mentor, because these terms imply differing status. We wanted the 
CTE teachers to be full partners, to stay firmly grounded in their specialty, and to teach math 
where it contributed to the learning of occupational skills. 

Discussions in the focus groups at each SLMP indicated that the process produced the 
intended result. Several math teachers said that their participation in this study increased their 
understanding and respect for CTE. Some began using the examples developed for the CTE 
courses in their own classes. For the first time they had a ready answer for the perennial student 
question, “How am I ever going to use this?” 

The CTE teachers benefited from this process by learning math in a collegial relationship. 
They were not “taught” math; instead they worked with an informed partner to improve their 
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ability to teach math. They were motivated to learn this math because it was part of their 
curriculum. Hill, Rowan, and Ball (2005) found that improving teachers’ mathematical 
knowledge can improve students’ math achievement. Furthermore, a review of over 400 
evaluation studies of mathematics and science curriculum and professional development models 
found that professional development that is tied to knowledge of the subject matter and/or how 
students learn the subject is more effective in terms of improving student achievement than is 
professional development that focuses only on teaching behaviors (Clewell, de Cohen, Campbell, 
& Perlman, 2004). The products of this relationship were the lesson plans, but just as important, 
CTE teachers who had a fuller understanding of the math they taught. 

Communities of Practice and Issues of Reform 

The most important of these core principles is to develop and sustain a community of 
practice, and this does have implications for the broader issue of school reform. Despite the wide 
usage of communities of practice, there is to our knowledge no generally accepted definition of 
this term. Wenger (1998) has provided the most developed theory of communities of practice, 
and on his Web site (http://www.ewenger.com/theory/index.htm) he defines them as follows: 
“Communities of practice are groups of people who share a concern or a passion for something 
they do and leam how to do it better as they interact regularly” (Wenger, 2005). He further 
refines this broad definition by the following characteristics: 

1. The domain: A community of practice is not merely a club of friends or a network of 
connections between people. It has an identity defined by a shared domain of interest. 
Membership therefore implies a commitment to the domain, and therefore a shared 
competence that distinguishes members from other people. 

2. The community: In pursuing their interest in their domain, members engage in joint 
activities and discussions, help each other, and share information. They build 
relationships that enable them to learn from each other. 

3. The practice: A community of practice is not merely a community of interest— people 
who like certain kinds of movies, for instance. Members of a community of practice are 
practitioners. They develop a shared repertoire of resources: experiences, stories, tools, 
ways of addressing recurring problems — in short, a shared practice. This takes time and 
sustained interaction. 

We prefer the term communities of practice to the one that is more frequently encountered in 
the education 1 itcraturc, professional learning communities (Hord 1997), because of its broader, 
more diffuse implications. Professional learning communities have all the same defining 
attributes as communities of practice, but typically refer to faculty within a given school working 
under the leadership of that school’s principal. The communities that emerged in our study were 
not from one school, and their principals were not involved in their activities. Our communities 
lacked what most of the literature identifies as essential for school reform (e.g., strong, consistent 
leadership) but they were able to improve measured performance in the academic area, 
mathematics, where the performance gap among students is most pronounced. This may be 
because of the presence in our study of many elements of successful curriculum integration 
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efforts, as defined by Johnson et al. (2003) and enumerated in Chapter 2. There is supporting 
evidence from at least one other cross-site study that shows such collaboration between 
mathematics and vocational teachers can result in successful curriculum integration under similar 
conditions (e.g., regular meetings and reflective, team writing of the lessons) (Hernandez & 
Brandefur, 2003). 

The typical school reform model focuses on the individual school, and leadership at the 
school level is the dimension most frequently cited as critical to whether a reform succeeds. Our 
communities cut across schools, school districts and even state borders, connecting teachers 
within a discipline who provided their own leadership. They worked within a pedagogic structure 
developed by the NRCCTE and facilitated by local research staff, but the communities of 
teachers made the critical decisions. It was the communities who selected the math concepts to 
be emphasized, developed the lessons, and decided when these lessons would be delivered. 

Fullan (2000) argues that people will not share knowledge with each other unless they feel 
morally compelled to do so. They must believe in the purpose. Furthermore, people will not 
share knowledge unless the dynamics favor change. Communities of professional practice have 
either a purpose or mission — to raise students’ achievement — and a focus on trying a new way of 
doing things. Finally, Fullan (2000) asserts “data without relationships leads to an information 
glut” (p. 6). The number of curricula and lessons that have been developed to date and sit on 
educators’ bookshelves is overwhelming. But the teachers who participated in the creation and 
revision of their lessons in professional, supportive, mutually beneficial relationships are not 
likely to simply cast them aside as just another new reform. The relationships in the community 
of practice helped to make the lessons part of the teachers’ repertoire. 

We make no claim that we reformed the schools in which our study was conducted. Even the 
colleagues of our teachers had no real knowledge of what was occurring in the experimental 
classes. This was by design. To avoid contamination of the design, our teachers did not share the 
lessons they had developed even when they were asked directly to do so by their colleagues. We 
do not know if communities of practice would emerge among teachers who are not as self- 
selected, but Borman et al. (2005) found the extent of participation in learning communities to be 
the strongest predictor of improved math achievement in their evaluation of Urban Systemic 
Initiative funded by the National Science Foundation. Their evaluation focused on four large 
cities and collected multiple pieces of information, including scores on mathematics achievement 
tests and teacher and student surveys. Regression analyses found those schools where the 
teachers reported the most involvement in learning communities to have the highest gains on 
student test scores. Borman et al. (2005) interpret this finding to indicate the importance of a 
supportive school culture. We believe that a supportive disciplinary culture is also critical; our 
study suggests that pedagogical change does not have to be top-down within a school but can 
occur within and across classrooms. 

This research project drew teachers from many schools and certainly did not influence the 
leadership or culture in any of them (though principals were aware of the math and CTE 
teachers’ involvement in the study). Nevertheless, simply bringing teachers together to teach 
them a new pedagogical model and giving them the authority to decide what concepts they 
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would incorporate and when and how to teach them led to significant differences in the 
performance of their students in a defined content area. Such a strategy does not require 
exceptional leadership or cultural change and thus may have broader potential as a school reform 
strategy. 



Future Research — Remaining Issues 

As with any research study, these results raise other questions. We elucidate key questions or 
issues that have arisen directly from the study and indirectly through requests for technical 
assistance. 

The first set of questions and issues relates to the process of bringing this approach to scale. 
A key research question emerges from the use of a volunteer sample in this study. We can 
assume that teachers who volunteer to participate — and to stick with a study like this for more 
than a year — are different from other teachers in important ways. Marketing specialists and 
others often describe the general population in terms of the rate at which they will accept a new 
product or innovation. This curve of innovation (E. M. Rogers, 1962, 1976) posits that roughly 
2.5% of the general population are innovators, the next 13% on the curve are early adopters, the 
middle 68% of the curve are early and late majority and the remaining part of the curve are 
laggards. Innovators and early adopters are very different from others on the innovation curve. 
We believe that the teachers who worked with us on this study were on the left side of the 
curve — the innovators and early adopters. The important next question in bringing a reform to 
scale is to what extent will the model work with the middle part of the curve, the early- and late- 
majority teachers? These are good teachers, competent professionals, but not readily inclined to 
embrace reform and innovation. 

A related question is the extent to which teachers have truly accepted a changed approach to 
teaching CTE. As part of this study, teachers were provided financial incentives, professional 
development, various kinds of support, materials, and recognition. What happens when those 
incentives and support are not offered? This is another key issue in school reform literature, that 
of reform sustainability. As many of our teachers told us, the educational landscape is littered 
with reforms that held promise but never really became part of the fabric of the school or the 
classroom. 

A third question that arises is a challenge to our community-of-practice core principle. 

Simple logistics and bureaucratic inertia militate against building such communities within 
school districts. This is partly a function of the critical mass issue raised earlier. Because many 
CTE teachers teach in single-teacher programs within comprehensive high schools or at regional 
CTE centers, assembling a community of practice following our guidelines will require cross- 
district, perhaps even cross-state cooperation. The question that arises from this study is the 
extent to which communities of practice can cut across these SLMPs, e.g., mixing health and 
auto teachers and maintaining viable communities of practice. 

A fourth question that merits investigation is a combination of two issues we observed in our 
study. By design, we focused on single CTE classes. As discussed in the report, with a relatively 
minimal time investment we were able to produce substantial improvements in math 
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achievement. The natural follow up question is the extent to which enhancing the math within an 
entire program or sequence of CTE courses affects math achievement. Several teachers 
suggested this as a solution to one of the kinds of problems that arose during implementation that 
we identified as the tipping point. That is, how much can you modify the CTE curriculum before 
it begins to more resemble a traditional math class and less resemble a CTE class, and thus risk 
losing students from this elective area? One response to the tipping-point issue is to spread the 
math throughout the program, which will also provide more math learning opportunities. 

In this study, we focused on in-service, professional development. We can envision a longer- 
term study directed at preservice education. How can we move this model into teacher education 
programs with the well-established silos of expertise? 

No doubt, there are many more lines of inquiry a reader of this report can envision. We 
welcome all suggestions and recommendations for ways in which we might continue to work 
with CTE teachers to improve their ability to equip their students with the skills necessary for a 
successful future. 
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TECHNICAL NOTES 
1. Site/SLMP-Specific Recruiting Procedures 

Direct mail and e-mail were used at SLMP A 14 to recruit CTE teachers. To support the direct 
mail and e-mail, an associate director for the state department of education and the director of K- 
16 initiatives for the Board of Regents sent e-mails to all high school principals, directors of 
career centers, and Tech Prep coordinators to inform them about the study and to ask them to 
encourage their teachers to participate. The staff at SLMP A made personal presentations about 
the study to three summer workshops for Tech Prep instructors. When these measures yielded a 
low response, telephone calls were made to all teachers in the occupational area in one state to 
request their participation 

SLMP B focused on teachers who taught programs that were certified by an industry- 
sponsored accreditation board for that specific type of CTE program. To start the recruitment, the 
board’s manager for one region of the nation sent an e-mail to the managers for all the states in 
his region asking them to encourage the teachers in their states to take part. The SLMP staff 
followed up with these state-level managers and provided them with applications for all the 
teachers who expressed interest. 

At SLMP C, the state supervisor for the occupational area was instrumental in the 
recruitment of teachers. She sent a letter of support for the study, which strongly encouraged 
teachers to participate. One of our project’s staff in the state developed a list of 25 
teachers/schools considered likely to participate and contacted them personally. A presentation 
about the study during a summer conference for CTE teachers by the NRC director yielded a 
number of applications. 

SLMP D followed a procedure similar to that of SLMP L. Staff made presentations about the 
study to conferences of CTE teachers to request their participation. Applications were distributed 
to interested parties and follow-up e-mail and telephone calls were made to encourage 
submission of the applications. 15 

A presentation by the NRC director at a conference of CTE educators at SLMP E helped to 
increase the pool of interested teachers, which our staff in that state had previously recruited. 
They had focused their initial efforts on the school districts of a major city and its western 
suburbs; they had special access because both had a history of involvement in the school district. 
The staff members also worked with professional associations and made personal telephone calls 
to local CTE directors to inform them of the study and to ask them to encourage their teachers to 
apply. 

Staff from SLMP F met with CTE instructors at their district conferences in five 
administrative districts of the state. A presentation was made that described the proposed study. 



14 Participants in this study were promised anonymity regarding the outcomes at their individual sites. To provide 
this anonymity, we refer to the six separate sites by capital letters, A to F. 

15 This site participated in the one semester study but not in the full year study. 
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Teachers who expressed an interest were given an application. To reach a pool of 40 interested 
teachers, follow-up telephone calls were made to selected teachers per recommendation of 
program specialists for the five administrative districts. 

2. Attrition Analysis 

Of the 229 teachers who submitted applications at the onset of the study, 198 (86.5%) 
participated in either the experimental or control groups of the pilot study during spring 2004. 
Attrition between assignment and participation in the pilot was over three times higher in the 
experimental group (24 to 7) primarily because of the professional development requirement of 
several days during the summer. Those who withdrew did so by not attending the professional 
development scheduled for their replication. The attrition among the control teachers was due to 
their assignment. When they were informed that they had been assigned to the control group, a 
small number decided they did not want to participate in the study because they had wanted to be 
in the experimental group. Importantly, there was no attrition of any of the teachers during the 
implementation of the pilot study. 

Of the 198 teachers who participated in the pilot, 131 (66.2%) also continued on to 
participate in the full-year study (see Table 18). Two factors explain most of the attrition during 
the pilot and full-year study. First, when we decided to test the effect of a 1-year intervention, 
only those classes offering a full year of curriculum were included, thus semester only or 
trimester only classes were dropped. A second important cause of the loss of teachers at this 
stage was the late decision to conduct the full-year study. The teachers were informed of this 
decision in May 2004, almost at the end of the intervention phase of the pilot study. They were 
invited to continue for the full year with the understanding that they would have to take part in 5 
days of professional development during the summer of 2004 and five more during the course of 
the study. When the dates were set for the professional development sessions, a number of 
teachers indicated they had conflicts with other commitments that they had previously made. 
With the substantial loss of experimental teachers between the pilot and full-year studies, and the 
higher-than-anticipated expenses of the pilot, the decision was made to randomly drop some of 
the control teachers to make the numbers in the two groups more nearly equal. A total of eight 
control teachers were randomly selected and informed that they would not be continued in the 
study for the full year, but they were invited to receive the promised professional development at 
the end of that year with the continuing control teachers. 

With as much attrition as the study experienced, the question arose as to whether those 
teachers who discontinued were systematically different from those who continued. We tested 
this question at two points: first, by comparing those who applied but did not participate in the 
pilot to those who did; and second, by comparing those who participated in the pilot only and 
those who participated in both the pilot and the full year. For the first comparison, limited 
individual data were available about those who applied but did not participate. What were 
available were the names of the high schools and career centers in which they taught. Using this 
information, data were assembled from the 2001-2002 Common Core of Data (CCD), compiled 
by the U.S. Department of Education, National Center for Education Statistics, to compare the 
characteristics of these schools. 



88 



National Research Center for Career and Technical Education 




Building Academic Skills In Context: Testing the Value of Enhanced Math Learning in CTE 



Table 18 



Number of Teachers Who Applied for the Pilot Study and the Number Who 
Participated in the Pilot Only, or in Both the Pilot and the Full-Year Study, by SLMP 



SLMP 


Applied 


Experimental 


Control 


Pilot 


Full-year 


Pilot 


Full-year 


A 


30 


8 


2 


13 


4 


B 


50 


22 


11 


22 


17 


C 


60 


28 


21 


27 


23 


E 


46 


18 


9 


22 


14 


F 


43 


18 


14 


20 


16 


Total 


229 


94 


57 


104 


74 



Note. To preserve participant anonymity, the order of presentation of these SLMP sites does not necessarily correspond with 
those in Figure 3. Site D was dropped after the pilot study for administrative reasons. 



Data were accessed from the CCD for 86 schools, 69 in which teachers administered the 
experimental intervention and 18 from which teachers applied but withdrew before participating. 
All but 12 of the 87 were comprehensive high schools; the CCD either did not provide or 
provided incomplete data for career centers. This is because most students in career centers 
attend on a half-day or other shared-time basis, and they are counted as part of the enrollments of 
their sending high schools, not of the centers themselves. The two sets of institutions (the 69 vs. 
the 18) were compared in terms of total enrollment, percentage of non-Hispanic European/Anglo 
enrollment, and percentage eligible for free or reduced-price lunches. The results are shown in 
Table 19. There were no significant differences between the schools of the participating teachers 
and the schools of those who withdrew. 

Table 19 



Comparison of School Enrollment Data for Experimented Teachers Who Participated in the 
Study and Those Assigned Who Withdrew 



Enrollment data 


Participated 
(n = 69) 


Withdrew 
(n = 18) 


t 


P 


M total school enrollment 
SD 


986.42 

710.04 


1084.68 

767.18 


.49 


.31 


% non-Hispanic European/ Anglo enrollment 
SD 


76.38 

22.00 


75.61 

22.99 


.12 


.45 


% eligible for free or reduced-price lunch 
SD 


28.50 

19.41 


25.86 

18.23 


.53 


.30 



These comparisons yield no information about the teachers, themselves, but they do imply 
that the characteristics of students in the classrooms of the participating and withdrawing 
teachers were similar. Separate analyses were done within the specific SLMPs if there were at 
least three schools with teachers who both participated and withdrew. Four such comparisons 
were possible, and no significant differences were found. 
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For the comparisons of the teachers who participated in the pilot but not in the full year, we 
had individual data, especially for what we assumed to be a critical characteristic: math anxiety. 
Tables 20 and 21 present information on gender, race/ethnicity, math anxiety, and annual income 
between teachers in the pilot only and those who continued for the full year. There were no 
differences in the two groups of experimental teachers. Control teachers who did not continue 
were more likely female and reported lower personal earnings. This suggests that the honorarium 
was not the primary reason these teachers continued through the full study. 

Table 20 



Characteristics of Teachers Who Withdrew After the Pilot Study and Those Who Participated in 
Both the Pilot and Full-Year Studies ( Chi-Square Analyses ) 



Characteristics 


Experimental Group 


Control Group 


Pilot 

study 

only 


Pilot + 
full-year 
studies 


r 


P 


Pilot 

study 

only 


Pilot + 
full-year 
studies 


r 


P 


Gender 


















% female 


41.1 


46.4 


.33 


.57 


59.5 


40.6 


3.45 


0.06 


% male 


58.9 


53.6 






43.2 


58.0 






Race/ethnicity 


















% non- White 


5.4 


14.3 


2.52 


.11 


5.7 


9.0 


.33 


0.56 


% White 


94.6 


85.7 






94.3 


91.0 






Base n for % 


56 


56 






35-37 


67-69 







Table 21 



Characteristics of Teachers Who Withdrew After the Pilot Study and Those Who Participated in 
Both the Pilot and Full-Year Studies ft tests) 



Characteristics 


Experimental 


Control 


Pilot 

only 


Pilot + 
full-year 


t 


P 


Pilot 

only 


Pilot + 
full-year 


t 


P 


Math anxiety 


















M 


19.94 


19.05 


1.11 


0.27 


17.90 


18.64 


0.63 


0.54 


SD 


3.61 


3.89 






4.92 


3.47 






Teaching income 


















M 


3.50 


3.36 


0.75 


0.46 


3.00 


3.49 


2.94 


0.01 


SD 


1.34 


1.49 






1.39 


1.46 






Base n for Ms 


34 


59 






20 


79 







Note. Math anxiety was measured on a scale from 6 to 24 (6 = highest anxiety, 24 = least anxiety). Annual income from teaching 
was reported at six levels (1 = less than $19,999, 2 = $20,000-$34,999, 3 = $35,000-$49,999, 4 = $50, 000-$7 4,999, 5 = 

$75, 000-$99, 999, 6 = $100,000 or more). 



In addition to these comparisons, we had one other way of examining the similarity of the 
students of continuing and noncontinuing teachers. Pretesting had been done with the classes of 
10 teachers whose course lasted only one semester. We compared the pretest scores of these 
classes with those of the 131 classes that were used to test the effect of the intervention. Table 22 
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presents the results. The pretest scores of the one-semester classes are higher in both the 
experimental and control groups, but the differences are not significant. 

Table 22 

Comparison of Pretest Scores of Classes Used to Test Effects of Experimented Interx’ention With 
One-Semester Classes That Were Discontinued 



Full-study One-semester only 



Group 


M 


SD 


n 


M 


SD 


n 


t 


P 


Experimental 


46.74 


9.81 


57 


52.46 


8.70 


4 


1.26 


.11 


Control 


49.13 


10.02 


74 


56.63 


15.25 


6 


1.18 


.12 



While the study experienced a high rate of attrition among its teachers, virtually all the 
analyses we conducted indicate that the teachers who continued were similar to those who 
withdrew on measured variables. We also know from poststudy interviews that of the 37 
experimental teachers who did not continue on for the full-year test of the intervention: 

• 11 cited schedule conflicts with the new professional development requirement. 

• 9 candidly acknowledged it was too much work. 

• 8 cited personal conflicts including a new baby, retirement, death in the family, moved to 
a new school, or were no longer teaching the same subject. 

• 6 teachers were teaching a semester or trimester length class. 

• 3 teachers did not participate for unknown reasons. 

3. Changes from the Pilot to the Full- Year Study 

The full-year study was implemented in five of the original six occupational areas, due to 
administrative difficulties in the sixth. A review of the formative evaluation data assembled as 
part of the pilot, one-semester study led to a number of changes in preparation for the full-year 
study. These included revisions in the pedagogic framework (the seven elements) and in the 
amount and kind of math support provided to the CTE instructors. The revisions in the model 
emphasized more bridging between the CTE and mathematics vocabulary, increased attention to 
how the embedded math is represented in traditional math instruction, and inclusion of formal 
assessment as element seven. Increased math support was provided through: lesson critique and 
practice in the professional development workshops; improved processes and guidelines for the 
preteaching and postteaching meetings; math cluster meetings of small groups between the 
workshops; Web sites with resources for use by teachers and facilitators in each of the 
occupational areas; and a reporting system for monitoring the implementation of the lessons by 
the CTE instructors. 
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APPENDIX A: LESSON EXAMPLES 



The Pythagorean Theorem 
Math Enhancement in the Building Trades 



Objectives: 

1. Students will demonstrate a working knowledge of Pythagorean theorem applications in 
rough framing, basic wall framing, footings, and foundations. 



2. Students will demonstrate a working knowledge of the Pythagorean theorem in 
traditional math problems. 



Element 1 - Teacher thinks out loud about building a wall frame and how to ensure that it truly 
contains 90-degree (square) comers. The same type of knowledge would be useful in 
constructing window frames, doorframes, and even constructing whole buildings. 



Element 2 - Teacher and students will brainstorm about what defines a rectangle and how to 
make sure that a rectangle truly has square corners (note: rectangles without square comers are 
not rectangles by definition). Student suggestions may include opposite sides of equal length, 90- 
degree angles, etc. Brainstorming will then move to how we can prove that the angles are truly 
square (diagonals the same length, i.e., the Pythagorean theorem). 



Teacher will ask questions to find out what students already know. 

• What can you tell me about linear measurements? 

• Does 3:4:5 mean anything to you? 

• Have you used this before? If so, how? 



Element 3 - Teacher will demonstrate how the 3:4:5 right angle is used as a shortcut and the 
steps in solving the generic formula ( a 2 + b 1 = c 2 ). 

Element 4 - Teacher will explain the Pythagorean theorem, and show them how it applies, using 
the terminology of math. (Work out the example problem/demonstrate how a wall frame’s 
rectangular shape would be ensured.) 

Key math terms: sides, legs, hypotenuse, diagonal, unknown, variable 

Element 5 - Students will try other examples similar to the first one presented. Hand out 
attached worksheet. Students should work out the first two problems individually. When they 
have finished the first two problems, they may compare, discuss, and correct their answers. The 
remaining two problems may be completed individually or in groups. 
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Element 6 - Students will explain or demonstrate what they did, to show understanding. 
Problems 3 and 4 on the worksheet. 

Element 7 - Students will be challenged to create their own math/CTE examples (estimating 
material orders, determining roof rafter lengths, etc.). Answer Problem 5 on the worksheet. 

[Special needs students could be given sheets with multiples of the 3:4:5 ratio, and will be 
allowed to use a calculator.] 
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Practice with the Pythagorean Theorem 



1. Explain how you would ensure that a standard exterior door opening (3'0" wide x 6'8" 
high) is square while constructing a wall frame. 



2. A residential building is 24' wide and 48' long. How long should the diagonals be? Show 
how cr + b 1 = c 1 can be applied to solve this problem. 



3. Determine the width of the rectangle (x) using the information shown: 




4. If a right triangle has a 10' base and a 6' height, how long is its hypotenuse? 



5. Name an example of where the Pythagorean Theorem could be of use (in math, CTE, or 
outside the classroom), other than those given in class. Hint: Where do you see right 
angles in the world around you? 
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Math-in-CTE Lesson Plan 



Lesson title: Manipulating the variable to change compression ratios Lesson no.: 10 


Occupational area: Automotive Technology 


CTE concept(s): Compression ratio 


Math concept(s): Direct/indirect variation; exponents. 


Lesson objective: 


Students will demonstrate a working knowledge of 
how to use direct and indirect variation and its 
application in automotive technology, while 
recognizing it in other contexts. 


Supplies needed: 


HP Books Auto Math Handbook by John Lawlor 
Copyright 1972, ISBN 1-55788-020-4 

Mathematics Dictionary and Handbook by Eugene 
Nichols and Sharon Schwartz 
Copyright 1999, ISBN 1-882269-07-1 


THE “7 ELEMENTS” 


TEACHER NOTES 
(and answer key) 


1. Introduce the CTE lesson. 

Two 5.0-L Mustangs are preparing to 
race. The driver of the black Mustang 
has increased the piston 
displacement, while the driver of the 
white Mustang has increased the 
chamber volume. Assuming every 
other variable is the same, which 
Mustang would you expect to win the 
race? 


This lesson will follow the lesson deeding with the 
calculation of compression ratios. 

The problems that you are solving here would use the 
same procedure that you used in the previous lesson. 
Please see notes in that lesson for assistance. 


2. Assess students’ math awareness 

as it relates to the CTE lesson. 

1 . What affects compression 
ratio? 

2. Why is compression ratio 
important when you are 
rebuilding an engine? 

3. What happens to the 
compression ratio if the piston 
displacement is increased and 
the chamber volume stays the 


Compression Ratio ( CR ) = 

Piston Displacement (PD) +ChamberVolume ( CV ) 


Chamber Volume (CV) 

Direct Variation - when two variables in an 
expression increase or decrease together. 

Indirect Variation - when an increase in one variable 
in an expression causes another variable in the equation 
change to decrease, and vice versa. 
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same? What is the term for 
this? 

4. What happens to the 
compression ratio if the 
chamber volume is raised and 
the piston displacement stays 
the same? What is the name for 
this? 

5. What is the difference between 
direct variation and indirect 
variation? 


NOTE: Try to make the concepts as simple as possible 
and relate the concept to what the kids already know. A 
3" diameter can vs. a 4" diameter can; which one holds 
more? The bigger the piston chamber, the bigger the 
volume you have to fill. The piston is traveling the 
same up/down distance, so the ratio gets larger 
because of the larger volume. 


3. Work through the math example 


For these two problems, you will need to calculate the 


embedded in the CTE lesson. 


compression ratios, and then compare them to the 


1. The compression ratio of a 5.0- 


original compression ratios. 


L Mustang with a 302 CID 


1. 


(cubic inch displacement), a 


Compression Ratio ( CR ) = 


piston displacement of 618.60 
cc, and a chamber volume of 


Piston Displacement (PD) + ChamberVolume (CV ) 


75 cc is 9.25:1. If the driver 


ChamberVolume (CV) 


would change the piston 
displacement to 647.70, would 


CR= 647.70 cc + 75 cc 


the compression ratio increase 


75 cc 


or decrease? Find the new 


CR = 722.7 cc 


compression ratio. 


75 cc 


2. The compression ratio of a 5.0- 


CR = 9.6:1 

The compression ratio will increase. 
2. 


L Mustang with a 302 CID, a 
piston displacement of 618.60 


Compression Ratio (CR) = 


cc, and a chamber volume of 


Piston Displacement (PD) + ChamberVolume (CV ) 


68 cc is 10:1. If the driver 


ChamberVolume (CV) 


would change the chamber 
volume to 80 cc, would the 


CR= 618.60 cc + 80 cc 


compression ratio increase or 


80 cc 


decrease? 


CR = 698.60 cc 
80 cc 

CR = 8.7:1 

The compression ratio will decrease. 
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4. Work through related, contextual 
math-in-CTE examples. 

1. Given the formula , 

HP = dxw 
33,000 

what happens to HP (horse 
power) as the d (distance) 
increases? Is this direct or indirect 
variation? 

w 

2. Given the formula E = — , 

what happens to the volts (E) 
when the watts (W) is 
increased? 


1. The horsepower will increase as the distance 
increases. This is direct variation. 

2. The volts will increase when the watts are 
increased. 


5. Work through traditional math 


1. Seeing that the R is in the denominator of the 


examples. 


fraction, it varies indirectly with I. 


E 

1. Given the equation I = — , 

R 




I varies (directly or indirectly) 




with R? Circle one. 




E 

2. Given the equation R = — , 




/ 


2. Seeing that the E is in the numerator of the 


R varies (directly or 


fraction, it varies directly with R. 


indirectly) with E? Circle one. 




6. Students demonstrate their 




understanding. 


Use the worksheet and answer key provided by North 


See supplemental worksheet with 


Montco Technical Career Center. 


Lesson #9. 




7. Formal assessment. 




1. Using the Ohm’s law formula 


1. The ohms will decrease because this is indirect 


E 


variation. 


R = — , what happens to the 




ohms (R) when the amps (I) 




are increased? Is this direct or 




indirect variation? 




P 

2. Assume that — = k, does P 




T 


2. Yes, P varies directly; and it will increase. 


vary directly with k? Will it 




increase or decrease? 
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3. A 284-CID engine has a 


3. 


cylinder volume (piston 


Compression Ratio ( CR ) = 


displacement) of 610.0 cc and 
a combustion chamber 


Piston Displacement (PD) +ChamberVolume ( CV ) 


Chamber Volume (CV) 


volume of 35 cc. What is the 


compression ratio? If we 


CR = 610.0 cc + 35 cc 


increase the chamber volume, 


80 cc 


will the compression ratio 




increase or decrease from the 


CR = 645 cc 


value just calculated? 


35 cc 


Calculate the compression 




ratio with a chamber volume 


CR= 18.4:1 


of 42 cc, leaving everything 


The compression ratio will decrease if the chamber 


else the same. 


volume is increased. 




CR = 610.0 cc + 42 cc 




80 cc 




CR = 645 cc 




42 cc 




CR= 15.4:1 
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Math-in-CTE Lesson Plan 



Lesson title: Maximizing profits 



Lesson no.: #37 



Occupational area: Business and Marketing 



CTE concept(s): Breakeven points 



Math concepts: Solving algebraic equations & linear programming 



Lesson objective: 


After completion of this lesson, the student should be 
able to compute low-level breakeven points and 
complex breakeven points using linear programming. 


Supplies needed: 


Linear Programming Paper 
Linear Programming Worksheet 
Linear Programming Worksheet Answer Sheet 
Graphing Review Sheet (if needed) 



THE “7 ELEMENTS” 



TEACHER NOTES 
(and answer key) 



1. Introduce the CTE lesson. 

Many manufacturers produce several 
products; they need to make enough of 
each product to meet their customers' 
needs and best utilize materials, labor, 
and machinery. 

As a manufacturer, you need to decide 
which products to produce that will 
maximize your profits. This is done 
through a process called linear 
programming. 

Before we can discuss maximum profits, 
we must first find out when the 
manufacturer will begin making a profit. 
This is called the breakeven point. 



Discuss the definitions of: 

Maximize: 

- To increase or make as great as possible 

- To find the largest value of a function 

Linear programming: A mathematical technique 
used in economics; finds the maximum or 
minimum of linear functions in many variables 
subject to constraints 

Breakeven point: The point, especially the level of 
sales of a good or service, at which the return on 
investment is exactly equal to the amount invested. 

(Breakeven point has been discussed previously.) 



2. Assess students’ math awareness as 
it relates to the CTE lesson. 

What is the formula for breakeven point? 

Try this problem: We are making 
widgets: variable costs are $3 per widget; 
fixed costs total $18,000; and we are 
selling them for $15 each. What would 
the breakeven point be? 



To assess student’s math background, give a short 
warm-up quiz with questions relating to breakeven. 

Breakeven point: Income = Expenses 

Equation: 3x + 18,000 = 15x 

-3x -3x 

lMQ0=Ux 
12 12 

x = 1500 widgets need to be sold to make a profit. 
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3. Work through the math example 
embedded in the CTE lesson. 

Otto Toyom builds toy cars and toy 
trucks. Each car needs 4 wheels, 2 seats, 
and 1 gas tank. Each truck needs 6 
wheels, 1 seat, and 3 gas tanks. His 
storeroom has 36 wheels, 14 seats, and 
15 gas tanks. He makes $1.00 on each car 
and $1.00 on each truck he sells. 

What combination of cars and trucks will 
maximize Otto’s profit? 

A few steps we must take to solve this 
problem: 

1. Pick variables. Let x = # cars and 
y = # trucks. 

2. Write inequalities that model each 

constraint. 

3. Graph each constraint. 

4. Calculate intersections of each 
inequality (using systems of 
equations). 

5. Write a profit equation. 

6. 6. Substitute each ordered pair from 
Step #4 into the profit equation from 
step #5. Determine which ordered 
pair has the highest profit. This is 
the Linear Combination that 
maximizes the profit! 



Linear programming paper has been included; it is 
nice to be able to do the problem at the top of the 
page, while having the graph paper included. 

Constraint: The state of being restricted or 
confined within prescribed bounds. 

KEY (numbering is associated with the steps in 
the left column): 

2. Wheels [Ax + 6y < 36] 

In English, this equation says that each car needs 4 
wheels, each truck needs 6 wheels, and together 
they can not add up to be more than 36 (which is 
the number of wheels Otto can hold in his 
storeroom). 

Seats [2x + y < 14] 

Gas tanks [x + 3y < 15] 

3. Graphing inequalities will need to be reviewed 
with students. (See attached sheet for further 
review). 

Students will also need to understand why we are 
looking at graphs only in the first quadrant (where x 
and y are both positive). Explain that there is no 
way for Otto to make a negative number of either 
cars or trucks, thus x and y must both be positive. 

4. Students must know that ANY maximum or 
minimum will occur at one of the vertices. (These 
are the intersections of the inequalities). Solving 
systems of equations will need to be reviewed with 
students. (See attached sheet for further review.) 

Intersections: (6, 2), (3, 4), (5.4, 3.2)-> use (5, 3), 
since you can’t make a fraction of a vehicle. 

5. Profit = lx+ 1 y 

6 . Max profit: 

(6, 2) Profit = $8 
(3, 4) Profit = $7 
(5, 3) Profit = $8 

So Max profit occurs either with 6 cars and 2 trucks 
or 5 cars and 3 trucks. 
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4. Work through related, contextual 
math-in-CTE examples. 

Truck drivers have just become popular 
because of a new TV series called “Big 
Red Ed.” Toy trucks are a hot item. Otto 
can now make $2.00 per truck, though he 
stills gets $1.00 per car. He has hired you 
as a consultant to advise him, and your 
salary is a percentage of the total profits. 
What is his best choice for the number of 
cars and the number of trucks to make 
now? How can you be sure? Explain. 


KEY: 

Profit = lx + 2 y 
(6, 2) Profit = $10 
(3, 4) Profit = $11 
(5, 3) Profit = $11 

So now, the Max profit occurs either with 3 cars 
and 4 trucks, or 5 cars and 3 trucks. 


5. Work through traditional math 
examples. 

The Smiths have a small farm where they 
grow corn and tomatoes for sale. They 
have a total of 21 acres available for 
planting. Because they cannot afford to 
pay a lot for help, they have many 
restrictions based on their labor. They 
have a total of 150 hours available for 
planting and 130 hours available for 
harvesting. Each acre of corn takes 6 
hours to plant and 4 hours to harvest. 
Each acre of tomatoes takes 10 hours to 
plant and 10 hours to harvest. A local 
grocery chain has agreed to purchase 3 
acres’ worth of tomatoes. 

If each acre of com can be sold for $600 
and each acre of tomatoes can be sold for 
$800, how many acres of each type 
should the Smiths plant? 


Use linear programming paper for this work. 

1. Let x = the number of acres of com planted and 
y = the number of acres of tomatoes planted. 

2. Write inequalities which represent the 
restrictions on 

The land available = [x + y < 21] 

The time available for planting = [6.r + lOy < 150] 
The time available for harvesting = [4x + lOy < 
130] 

The arrangement made with the grocery chain = [v 
> 3] 

3-4. Graph the system you found in part (a) and 
find the vertices: vertices (x, y): (0, 3); (18, 3): (15, 
6); (10, 9); (0, 13) 

5. Profit = 600x + 800y 

6. Substitute each vertex into the profit equation: 

(0, 3) Profit = $2,400 

(18, 3) Profit = $13,200 
Max (15, 6) profit = $13,800 
Max (10, 9) profit = $13,200 
Max (0, 13) profit = $10,400 
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6. Students demonstrate their 
understanding. 

See Attached Linear Programming 
Worksheet. Use with linear programming 
paper. (This assignment may take several 
nights, as the problems can be 
challenging.) 


See Answer Sheets for completed graphs. 
ANSWERS: 

Oil: 

a. Profit = 30T + 15C 

b. T + C < 40,000; T + C > 18,000; 

6C + 2T < 120,000 

c. See graph. 

d. (Texas, Calif.) = (40,000, 0) 

Profit = $1,200,000 

Park: 

a. x = old member; y = new member 
Profit = 1 0,r + 8v 

b. x > 0, y > 0; x < 9, y < 8; 6 < x + y < 15, y > 3; y > 
1/2 x, y < 3x 

c. See graph. 

d. no, you must have old members because the 
graph isn’t shaded at x = 0 

e. (old, new) = (9, 6) Profit = $138 

f. (old, new) = (1.5, 4.5) but since we can’t have 0.5 
of a person, use (2, 4) Profit = $52 

g. Profit = lOx + 12 y (old, new) = (7, 8) Profit = 
$166 

Aircraft: 

a. x = Camel, y = Hippo Profit = 300x + 200y 

b- x + y <12 x - 11, y <7 ; y < 2x1 
100x + 200y >1000 

c. See graph. 

d. (Camel, Hippo) = (11, 1) Profit = $3,500 


7. Formal assessment. 

You are working at a small art store and your 
boss has decided to allow you sell some of 
your art. Your boss has determined that you 
can choose what combination of art you 
would like to sell, so that you can make the 
most profit possible. 


ANSWER: 
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1. Each pastel requires $5 in materials 
and earns a profit of $40. 


x = Pastel, y = Watercolor 


2. Each watercolor requires $15 in 
materials and earns a profit of $100. 


Profit = 40,r + 1 OOv 

Constraints: 5x + 15y < 180; x + y < 16; 


3. You have $180 to spend on 


x>3,y>2 


materials. 


Possible Max profits: (3, 11); (3, 2); (6, 10); 


4. You plan to make at least 3 pastels 


(14, 2) 


and at least 2 watercolors. 


Max (6, 10) Profit = $1,240 


5. You can make, at most, 16 pictures. 


What is the optimum number of pastels 
and watercolors that produces the 
maximum profit? 
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Review of Graphing Linear Inequalities: 

There are 2 ways to graph: 1. Re-write, in slope intercept form (y = mx + b) 
2. Solve for the .v-i intercept and y-intercept. 

Example from the Toy Problem: 4x + 6y < 36 



1 . 



V 

To Graph: 



4x + 6y < 36 

-_4x -4x 

6 y < -4x + 36 

6 6 



2 _ 

x + 6 

3 



1. Start at y-intercept (y = 6); put a dot 

2. From the y-intercept, use the slope 
another dot. 



f _2_\ 

3 



and go DOWN 2, RIGHT 3, put 



3. Continue with this pattern several more times; connect your dots. 



2. 4x + 6y < 36 

.v-intcrccpt: Let y = 0, and solve for x. 

4x + 6(0) = 36 
4x_= 36 
4 4 

x = 9 
(9, 0) 



y-intercept: Let .v = 0, and solve for y. 

4(0) + 6y = 36 
6y = 36 
6 6 
y = 6 
( 0 , 6 ) 



To Graph: Plot the points on your graph and connect the dots. 

(Remember to shade in the appropriate direction, based on inequality sign.) 

Review of solving systems of linear equations: 

There are 2 ways to solve: 1. Algebraically 2. Graphically (using TI graphing calculator) 
Example from the Toy Problem: 4x + 6y < 36 

2x + y < 14 

1. Algebraically: Eliminate one variable by multiplying one equation by a number that will make 
one coefficient be the opposite of one from the other equation. 

4x + 6y = 36 4x + 6y = 36 

-2(2* + y = 14) -4x - 2v = -28 

Next, add the two equations. 4y = 8 

Divide by 4. 4 4 

y = 2 



National Research Center for Career and Technical Education 



105 




Building Academic Skills in Context: Testing the Value of Enhanced Math Learning in CTE 



Now substitute this back into one equation to solve for the other variable: 

2x + (2) = 14 
x = 6 

2. Graphically: First, solve for y, to put in slope-intercept form (y = mx + b ) for each equation. 

2 

4x + 6y = 36 y = - —x +6 
3 

2x + y = 14 -» y = -2x + 14 

Next, plug both equations into the y = menu; and graph. Under the CALC menu, choose 
INTERSECT. Scroll close to the intersection point, and click ENTER (3 times), until the screen 
reads the INTERSECTION values. 
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Linear Programming 



NAME: 



Oil Refinery Problem: 

Sabrina Burmeister is chief mathematician for Pedro Leum’s Oil Refinery. Pedro can buy 
Texas oil, priced at $30 per barrel, and California oil, priced at $15 per barrel. He consults 
Sabrina to find out what is the most he might have to pay in a month for the oil the refinery uses. 

a. Define variables for the number of barrels of Texas oil, and then number of barrels of 
California oil, purchased in a month. Then write an equation expressing the total cost of the 
oil in terms of theses two variables. 

b. Sabrina finds the following restrictions on the amounts of oil that can be purchased in a 
month: 

i. The refinery can handle as much as 40,000 barrels per month. 

ii. To stay in business, the refinery must process at least 18,000 barrels a month. 

iii. California oil has 6 pounds of impurities per barrel. Texas oil has only 2 pounds of 
impurities per barrel. The most the refinery can handle is 120,000 pounds of 
impurities a month. 

Write a system of inequalities representing this information. 

c. Plot the graph of the system in Part b. 

d. What is the maximum feasible amount Pedro might have to spend in a month? How much of 
each kind of oil would yield this maximum cost? 
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Park Clean-Up Problem: 

Your marketing club has arranged to earn some extra money by cleaning up Carr Park. The 
city recreation department agrees to pay each old member $10 and each new member $8 for their 
services. (The club did the same thing last year, so the old members are experienced.) 

a. Define variables, then write an equation expressing the dollars the club earns in terms of the 
numbers of old and new members who work. 

b. The following facts restrict the numbers of students who can work: 

i. The number of old members and the number of new members are non-negative. 

ii. The club has, at most, 9 old members and, at most, 8 new members who can work. 

iii. The department will hire at least 6 students, but not more than 15. 

iv. There must be at least 3 new members. 

v. The number of new members must be at least _ the number of old members, but no 
more than 3 times the number of old members. Write inequalities for each of the 
above requirements. 

c. Draw a graph of the solution set of this system of inequalities. Remember that club members 
come only in integer quantities! 

d. Based on your graph, is it feasible to do without any old members at all? Explain. 

e. What numbers of old and new members would earn the maximum feasible amount? What 
would this amount be? 

f. What is the minimum feasible amount the club could earn? 

g. Suppose tradition was broken and the new members were paid more than the old members 
were. What would be the maximum feasible earnings if new members earn $12 and old 
members earn $10? 
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Aircraft Problem: 

Calvin Butterball is chief mathematician for Fly-By-Night Aircraft Corporation. He is 
responsible for mathematical analysis of the manufacturing of the company’s two models of 
planes, the Sopwith Camel and the larger Sopwith Hippopotamus. Fly-By-Night makes a profit 
of $300 per Camel and $200 per Hippo. 

a. Define variables for the number of Camels and the number of Hippos. Then write an equation 

expressing the total profit of the planes in terms of the two variables. 

b. Calvin finds the following restrictions for the production of these aircraft: 

i. Fly-By-Night can produce, at most, 12 aircraft per year. 

ii. They can manufacture no more than 1 1 Camels and no more than 7 Hippos in a year. 

iii. The number of Hippos can be, at most, 2 times the number of Camels. 

iv. A Hippo requires 200 hours to build, and a Camel requires 100 hours to build. The 
company will spend at least 1,000 hours working on the aircraft. Write a system of 
inequalities representing this information. 

c. Plot the graph of the system in Part b. 

d. How many Camels and Hippos should be produced to give the greatest feasible profit? What 
would this profit be? 
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Linear Programming Paper 

Name: 

Title: 
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APPENDIX B. TEST ITEM ANALYSIS 



TerraNova Math 
Concept 


Questions Addressing the Math Concept 
(X’s denote test questions covering the 
concept) 


Corresponding CTE-Math Lessons 




CTBS 

Basic 

Battery 

(Terra 

Nova) 


CTBS 

Survey 

(Terra 

Nova) 


Accuplacer 

(Elem 

Algebra 


Work 

Keys 


Info 

Tech 


Auto 

Tech 


Health 


Bus & 
Mktg 


Ag 

Mech 


Numbers and 
number relations 


Compare, 

order 


X 




X 






12,13, 

14,15 








Equivalent 

forms 


X 




X 






1 


7 






Percent 


X 


X 




X 


5,7 


4, 11 


6, 7, 8, 
10 


4,21, 
25, 30, 
32, 33, 
36, 40 


1,6 


Exponents, 

scientific 

notation 


X 


X 


X 




2, 4,5 


8,9, 10 








Number line 


X 




X 






12, 13 








Computation and 

numerical 

estimation 


Computation 


X 


X 


X 






I, 2, 3, 
4, 5, 6, 

II, 12, 
13, 14, 

15 








Computation 
in context 


X 


X 




X 


2, 3, 4, 
5, 6,7 


1, 2, 3, 
4, 5, 6, 
7, 11, 
12, 13, 
14, 15 


1, 3, 4, 6, 
7, 8, 11 


4, 15, 
17,21, 
30, 33, 
40,41 


1, 2, 3, 
4, 5, 6, 
7, 10, 
13, 14, 
15, 16 


Operation concepts 


Permutations, 

combinations 

Operation 

properties 


X 

X 




X 


X 


5 
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Measurement 


Estimate 










3 




6, 7, 10 






















18, 20, 


7, 12, 


Rate 








X 


3,7 




1,8 


25,32 


15 


Scale drawing. 


















7, 11, 


map, model 


X 


X 




X 


3,7 






27 


13 


Convert 


















1, 2, 3, 


measurement 


















4, 5, 6, 


units 


X 


X 




X 


3,4,7 




1,3,8 




14, 15 


Indirect 




















measurement 


X 


X 
















Use ruler 


X 








3 




3,8 


27 




Geometry and 
spatial sense 


Pythagorean 

theorem 


X 
















17 


Transformations 


X 


X 
















Apply 

geometric 

properties 


X 


X 










9 




7 


Geometric 

constructions 




















Data analysis, 
statistics and 
probability 


Interpret data 














1, 2, 3, 6, 




8, 9, 


display 


X 








1,6 




8, 10, 11 


4 


10, 12 


















3, 4, 
15, 18, 




Complete/ 
















21,25, 




construct data 










1, 4, 5, 




1, 2, 3, 5, 


27,31, 


8,9, 


display 










6 


11 


10, 11 


32,41 


10, 12 


Make inferences 
















4, 25, 




from data 


X 


X 










1,6, 11 


32 


8 


Evaluate 
conclusions 
drawn from data 


X 


X 
















Statistics 


X 


X 




X 


1,5,6 




7 


21, 31 




Probability 


X 


X 












36 




Use data to 
solve problems 


X 






X 








41 


9 


Compare data 


X 














32 




Describe, 
evaluate data 


X 


X 












25, 32 
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Patterns, 

functions, algebra 


Function 


X 


X 


X 








7 


3, 15, 
17, 18, 
27, 37, 
40 


7, 10 


Equation 


X 




X 




2,5 




7 


3, 15, 
17, 18, 
27, 37, 
40 


7, 10 


Inequality 


X 


X 


X 








7 






Graph 

quadratic 

equation 


X 


















Model 

problem 

situation 


X 


X 


X 




2 




7 


3, 15, 
18,27, 
37 


7, 10 


Use algebra to 
solve problems 






X 




3 


4, 5, 6, 
8 


7 


15, 17, 
18, 27, 
37,40 


7, 10 


Problem solving 
and reasoning 


Develop, 

explain 

strategy 


X 


X 
















Solve 

nonroutine 

problem 


X 


X 










11 






Proportional 

reasoning 


X 


X 








8, 9, 10 








Evaluate 

conjectures 




















Communication 


Model math 
situations 














5 






Make 

conjectures 














5 


20 




Evaluate ideas 
















20 




Explain 

thinking 
















20 




Explain 

solution 

process 




















Algebraic 

expressions 






X 






3 




15 
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Calculating 
perimeter/ 
area/volume of 
a rectangle, 
circle, triangle 








X 


3 


2,3,7 


4,8 




1,2,3, 
4, 5, 6, 
13, 14, 
15 


Calculating 
angles (trig) 


















16, 17 


Measuring 

angles 

(compass and 
protractor) 














9 







118 



National Research Center for Career and Technical Education 





Building Academic Skills In Context: Testing the Value of Enhanced Math Learning in CTE 



APPENDIX C. UNIVERSAL AGENDAS FOR PROFESSIONAL DEVELOPMENT 

Late-Summer 2004 Professional Development (PD) Session 
Welcome Back! 

• A word from the National Research Center 

• Bringing closure to Year 1 

• Tentative schedule for Year 2 

• Roles, Responsibilities, Incentives 
Math Concept Mapping for Year 2. 

• Review the math concepts addressed in Year 1. 

(Note: Be explicit about which Year 1 lessons addressed which concepts.) 

• Based on information gained from the curriculum/concept organizers, 

o What new math concepts should be added to “maximize the math” in Year 2? 
o What new lessons are needed to address the newly selected math concepts? 
o When in the school year should these new lessons be taught? 

• What math support is needed for Year 2? [SLMP-determined, generalized math 
instruction and use of technology tools can be added here.] 

The “ 7 elements Introducing the “improved” math enhancement process. 

• Introduce the lesson template for Year 2. 

• Walk through the 7 elements: Model/demonstrate actual lesson 
improvement/development . 

Critique of Year 1 Lessons. 

• In small groups, teachers identify what worked well, what requires improvement, what 
needs to be merged or eliminated. 

• Written feedback is prepared by groups for the creating team. 

• Key: All teachers who taught the lesson contribute copies of what they added to/changed. 
Lesson Development for Year 2. 

• CTE-math teacher-teams or clusters are formed to develop lessons for implementation in 
fall 2004, improving Year 1 lessons and developing new lessons to maximize the math. 

• Teachers are given time in the PD sessions to work together on the lessons. 

Presentation and practice of lessons improved and/or newly developed for fall 2004. (Locus is on 
practice .) 

• CTE-math teacher-teams are assigned times for teaching their lessons. 

Important: All teachers need to see cdl lessons taught. Possible options: 

o Ideal: Whole-group presentations, 1.5 hours/team; (one session for each 
concept — there may be more than 12 concepts), 
o Or, conduct “round robins,” ensuring that everyone gets a chance to learn about 
every lesson. 

• CTE-math teacher-teams practice teaching their lessons: 

1. Math teacher “preps” with the basic math instruction related to the lesson. 

2. CTE teacher teaches the lesson. 

• All teachers provide written feedback to the presenting team (more detailed than 
applause). 
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Wrap-Up Logistics: 

• Establish a communication strategy for the year. 

• Establish drop-dead dates for finishing fall lessons. 

• Announce to whom and in what form the finished fall lessons should be sent. 

• Review “assignments” for lesson development and presentation at the next PD. 

• Announce/review details for the fall math cluster meetings. 

• Announce/review details for the late fall PD session. 

• Review consent, pretesting, and survey procedures. 



Late- Fall 2004 Professional Development (PD) Session 

SLMP preparations for the Fall PD sessions: 

• Send out information on location, dates, times for PD. 

• Collect lesson plans in advance from teams who will present; have copies ready for PD. 

• Send out a “what to bring” list. Teachers should bring: 

o Samples of student work for Step 7: one for each lesson taught. PLEASE add 
your teacher ID and remove students’ names, 
o Video of your fall teaching tape (if scheduled before the fall PD), 
o Any changes made to the lesson plans you have already taught, 
o Anything you have added to the lesson plans, such as: additional worksheets, 
PowerPoints, test questions, class activities, etc. These contributions will be 
shared with the others and posted on the Web site, 
o Changes in your teaching schedule, scope and sequence; and plans about when to 
teach math-enhanced lessons. 

Agenda Items to Include: 

• Center Updates 

o Brief review of SLMP Year-1 results, 
o Any changes in research procedures or processes. 

• SLMP Updates 

o Overview of “how it’s going” 

■ How are the lessons going? 

■ Which lessons worked well? 

■ Which lessons need improvements? 

■ Issues and challenges? 

o Summaries of Math Cluster meetings 

■ What has been learned? 

■ What math support is needed for what lessons? 

■ (Schedule specific math instruction for group if needed), 
o Housekeeping (schedule whenever it makes sense to do this) 

■ New contracts and PVCs; IRB forms, if needed? 

■ Updated info sheets, if needed. 

■ Collect instructional artifacts from fall lessons, including videotapes 
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• Presentation and practice of lessons improved and/or newly developed for winter 
2004-2005. (Note: Teacher-teams were assigned these lessons in the summer PD and 
continue to develop them throughout the fall for presentation at the next PD.) 

o CTE-math teacher-teams are assigned times for teaching their lessons. Important: 
All teachers need to see ah lessons taught. Possible options: 

■ Ideal: Whole-group presentations, 1.5 hours/team;(one session for each 
concept — there may be more than 12 concepts). 

■ Or, conduct “round robins,” ensuring that everyone gets a chance to learn 
about every lesson. 

o CTE-math teacher-teams practice teaching their lessons: 

1. Math teacher “preps” with the basic math instruction related to the lesson. 

2. CTE teacher teaches the lesson. 

o All teachers provide written feedback to the presenting team (more detailed than 
applause). 

• Wrap-up 

o Establish drop-dead dates for finishing “winter” lessons. 

o Announce to whom and in what form the finished winter lessons should be sent, 
o Review “assignments” for lesson development and presentation at the next PD. 
o Announce/review details for the winter math cluster meetings, 
o Announce/review details for the late-winter PD session, 
o Review posttesting and survey schedule and procedures. 



Winter 2005 Professional Development (PD) Session 

SLMP preparations for the late- winter 2005 PD sessions: 

• Send out information on location, dates, times for PD. 

• Collect lesson plans in advance from teams who will present; have copies ready for PD. 

• Send out a “what to bring” list. Teachers should bring: 

o Important: Samples of student work for Step 7 = one for each lesson taught. 

Please add your teacher ID and remove students ’ names, 
o Your fall and/or spring teaching videotapes, 
o Any changes made to the lesson plans you have already taught, 
o Anything you have added to the lesson plans, such as: additional worksheets, 
PowerPoints, test questions, class activities, etc. These contributions will be 
shared with the others and posted on the Web site, 
o Changes in your teaching schedule, scope, and sequence; and plans about when to 
teach math-enhanced lessons. 

Agenda Items to Include: 

• Updates 

o The latest on Year-1 results and publications, 
o Any changes in research procedures or processes, 
o Overview of “how it’s going.” 

■ Overall, how are the lessons going? 
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■ Are lessons being taught as scheduled? 

■ What are the issues and challenges? 

• Presentation and practice of lessons (for spring 2005 and/or improved lessons from fall.) 

o CTE-math teacher-teams are assigned times for teaching their lessons. 

Important: All teachers need to see all lessons taught. Possible options: 

■ Ideal: Whole-group presentations (about 1 hour/team). 

■ Or, conduct “round robins,” ensuring that everyone gets a chance to learn 
about every lesson. 

o CTE-math teacher-teams practice teaching their lessons: 

1. Math teacher “preps” with the basic math instruction related to the lesson. 

2. CTE teacher teaches the lesson. 

o All teachers provide written feedback to the presenting team (more detail than 
applause; use the critique forms). 

• Maximizing the Math 

o Share specific ideas, strategies, techniques, resources for maximizing the math in 
the lessons. (This could be ideas from the math captains, or authors of the lessons, 
or other math experts related to the SLMP.) 
o Share summary of what was learned in math cluster meetings; as needed, provide 
specific math instruction related to concepts in the lessons to the whole group. 

• Improving Lessons for “Posterity” 

o Provide time and opportunity for teachers to make suggestions for improving the 
lessons taught thus far: 

■ What changes are needed, so that lessons may be used by other teachers in 
the future? 

■ What activities/examples/handouts could be added to address different 
levels of math ability? 

■ How can we further “maximize the math” in each lesson? (Refer back to 
previously shared ideas, strategies, techniques, and resources.) 

o Have teacher-teams (authors) spend time making improvements to their lessons. 

• Housekeeping (schedule whenever it makes sense to do this): 

o Have teachers complete new contracts and PVCs (for final payment in summer 
2005). 

o Update teacher information sheets, if needed. 

o Collect instructional artifacts to add to Web site and share with authors, 
o Collect videotapes and student samples, 
o Collect preteaching and postteaching reports, 
o Collect revised teaching schedules for master grids. 

• Wrap-Up 

o Establish drop-dead dates for finishing new and improved lessons, 
o Announce to whom and in what form the finished lessons should be sent, 
o Announce/review details for the spring math cluster meetings, 
o Announce/review details for the final summer 2005 PD/debriefing session (1 full 
day, followed by PD for control teachers), 
o Review posttesting and survey schedule and procedures; field questions. 
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Summer 2005 Professional Development Session 

Day 1 for Experimental Teachers 

• Center activities: 

o Research and debrief: Jim shares preliminary results 
o Focus groups 

• SLMP-level activities: 

o Final collection of teaching reports, teaching tapes, student samples 
o Celebration 

o Next steps for carrying this forward 
Days 2-3 for Control Teachers 

• Overview of the study; research debrief 

• Introduce the NRC Math Enhancement Model: Pedagogy _ Process 
(SLMP leaders and teacher-teams lead) 

o Walk-through of the concept mapping process 
o Walk-through of the 7 elements 
o Model lesson demonstrations 
o Introduction to the set of lesson plans 
o Talk about math-CTE teacher-partnerships 
o What it takes to implement the lessons 
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APPENDIX D. MATH-IN-CTE LESSON PLAN TEMPLATE AND RUBRIC 

Math-in-CTE Lesson Plan Template 



Lesson Title: 


Lesson no.: 


Occupational area: (example: Business and Marketing) 


CTE concept(s): (example: Breakeven points) 


Math concepts: (example: Solving algebraic equations) 


Lesson 

objective: 




Supplies 

needed: 





THE “7 ELEMENTS” 


TEACHER NOTES 
(and answer key) 


1. Introduce the CTE lesson. 




2. Assess students’ math awareness as it relates to the CTE 
lesson. 




3. Work through the math example embedded in the CTE 
lesson. 




4. Work through related, contextual math-in-CTE 
examples. 




5. Work through traditional math examples. 




6. Students demonstrate their understanding. 




7. Formal assessment. 
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Rubric for Critiquing Math-Enhanced Lessons 



Lesson Title: 



Lesson Number: 



As you review or obsen’e the lesson , please check the appropriate boxes in the rubric below. Use the comment box 
to make suggestions and recommendations. 



Elements 


Completed? 


Needs Improvement 


Comments 


1. Introduce the 
CTE lesson. 


_ Specific objectives of CTE 
lesson are explicit. 

_ Detailed script is provided 
for introducing lesson to 
students as a CTE lesson. 

_ Math concept(s) embedded 
in the CTE lesson is/are 
clearly identified. 

_ Script is provided to point 
out the math in the CTE 
lesson. 


_ Lesson objectives are 
unclear or not evident. 

_ Little or no script is 

provided for introducing 
lesson as a CTE lesson 
and/or lesson is/are 
introduced as a math 
lesson. 

_ Math concept(s) 

embedded in the CTE 
lesson is not made clear. 

_ Script is not provided to 
point out the math in the 
CTE lesson. 




2. Assess 

students’ math 
awareness as 
it relates to 
the CTE 
lesson. 


_ Lesson contains learning 
activities and/or well- 
developed questions that 
assess all students’ 
awareness of the embedded 
math concept. 

_ Math vocabulary and 

supporting instructional aids 
are provided to begin 
bridging of math to CTE. 


_ Lesson contains no 
learning activities and 
few, if any, questions 
that support assessment 
of all students' 
awareness of the 
embedded math 
concept. 

_ Math vocabulary and/or 
instructional aids are not 
provided. 




3. Work through 
the math 
example 
embedded in 
the CTE 
lesson. 


_ Script provides specific 
steps/processes for working 
through the embedded math 
example. 

_ CTE and math vocabularies 
are explicitly bridged in the 
script and supported with 
instructional strategies 
and/or aids. 


_ Steps/processes for 
working through the 
embedded math 
example are incomplete 
or missing. 

_ Little bridging of CTE 
and math vocabularies is 
scripted; few or no 
strategies and/or aids are 
provided to relate the 
CTE to math. 
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4. Work through 
the related, 
contextual 
examples. 


_ Lesson provides a work- 
through of examples, using 
the same embedded math 
concept in similar examples 
from the same occupational 
area. 

_ Example problems are at 
varying levels of difficulty, 
from basic to advanced. 

_ Script continues to bridge the 
CTE and math vocabularies, 
supported with instructional 
strategies and/or aids. 


_ Few or no additional 
examples of the 
embedded concept are 
provided. 

_ Examples do not reflect 
varying levels of 
difficulty. 

_ Little or no bridging of 
CTE and math 
vocabularies is evident 
in the script or 
supported with 
instructional strategies 
and/or aids. 




5. Work through 
traditional 
math 

examples. 


_ A variety of examples are 
scripted to illustrate the math 
concept as it is presented in 
traditional math tests. 

_ Examples move from basic 
to advanced. 

_ Script continues to bridge the 
CTE and math vocabularies, 
supported with instructional 
strategies and/or aids. 


_ Few or no math 

problems illustrate the 
math concept as it is 
presented in 
standardized tests. 

_ Examples do not reflect 
varying levels of 
difficulty. 

_ Little or no bridging of 
CTE and math 
vocabularies is evident 
in the script or 
supported with 
instructional strategies 
and/or aids. 




6. Students 
demonstrate 
understand- 
ing. 


_ Lesson provides learning 
activities, projects, etc., that 
give students opportunities to 
demonstrate what they have 
learned. 

_ Lesson ties math examples 
back to the CTE content; 
lesson ends on the CTE 
topic. 


_ No learning activities, 
projects, etc., provide 
students with 
opportunities to 
demonstrate what they 
have learned. 

_ Lesson fails to tie the 
math back to CTE or 
end on the CTE topic. 




7. Formal 
assessment. 


_ Lesson provides 

questions/problems that will 
be included in formal 
assessments (tests, projects, 
etc.) in the CTE unit/course. 


_ Example questions/ 
problems are not 
provided for use in 
formal assessments in 
the CTE unit/course. 
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APPENDIX F. MATH CAPTAIN MEETING FORMS 
SLMP Pre-Math Cluster Meeting Report Form 

SLMPs will summarize information from Preteaching and Postteaching Reports below, and 
forward as an e-mail attachment or by fax to math captains at least 1 week prior to each cluster 
meeting. SLMPs will coordinate with the math captains to arrange for submission of the 
videotape. 



Date sent: 


Lrom: (SLMP director) 


To: (math captain) 


Date of scheduled meeting: 


Location of meeting: 



Math-enhanced lessons that CTE teachers have reported they taught prior to this cluster 
meeting: 



CTE teachers and math-teacher partners from whom no preteaching or postteaching reports 
have been received, and why: (In advance of the cluster meeting, SLMPs will contact teachers 
who have not submitted their reports to determine why.) 



Math teachers’ reports of math concepts that CTE teachers needed the most assistance with: 

(from Question 6, Preteaching Report) 



Math teachers’ suggestions for improving the teaching of math concepts in these lessons: 

(from Question 7, Preteaching Report) 



CTE teachers’ suggestions for improving the teaching of the math concepts in these lessons: 

(from Question 9, Postteaching Report) 
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Math Captain Post-Math Cluster Meeting Report Form 

Math captains will submit this form as an e-mail attachment or by fax to SLMP facilitator 
within 1 week following each cluster meeting. SLMPs will contact math captains to arrange for 
submission of the videotape. 



Date submitted: 


From: (math captain) 


To: (SLMP director) 


Date of meeting: 


Location of meeting: 



Teachers attending: 



Lessons and math concepts discussed: 



Difficulties teachers reported in teaching these lessons and concepts: 



Assistance provided: 



Other problems discussed: 



Follow-up planned for teachers who did not attend: 



Suggestions for additional math support that the National Research Center or SLMP site 
should provide: 
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APPENDIX G. MATH TEACHER PRETEACHING REPORT FORM 

Submit as an e-mail attachment or by fax to your SLMP Director within 1 week following 
each preteaching review of a math-enhanced CTE lesson with your CTE instructor. 



Your ID #: 


CTE teacher’s ID #: 


Date of review: 


Title of lesson reviewed: 


Lesson number: 



Answer the following questions by putting an X in the box that best reflects your opinion on 
the scale following each question: 

1. In your judgment, how well are the math concepts 
integrated into the occupational content of this lesson? 



2. How adequate is the amount/depth of instruction in this 
lesson to teach students the math concepts? 



3. How would you rate the CTE instructor’s “comfort” 
with teaching the math in this lesson? 



4. How much assistance do you think you gave the CTE 
instructor? 



5. Are all 7 elements of the math enhancement model clearly presented in the lesson? 
Yes No_ If no, what elements are weak or missing? 



6. What part(s) of the math in this lesson did the CTE instructor need the most assistance with? 



7. Do you have any suggestions for improving the teaching of the math concepts in this lesson? 



Completely 



Completely 



High 



A lot 



Not at all 



Not at al 



Low 



None 
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APPENDIX H. CTE TEACHER POSTTEACHING REPORT FORM 

Submit as an e-mail attachment or by fax to your SLMP director within 1 week following 
each teaching of a math-enhanced CTE lesson. 



Your ID #: 


Math teacher’s ID #: 


Date(s) lesson taught: 


Title of lesson taught: 


Lesson no: 


Total class time, in minutes, spent on this lesson: 


Total number of classes in which the 
lesson was taught: 



Answer the following questions by putting an X in the box that best reflects your opinion on 
the scale following each question: 

1. In your judgment, how well were the math concepts 
integrated into the occupational content of this lesson? 



2. How adequate is the amount/depth of instruction in this 
lesson to teach students the math concepts? 



3. How would you rate your comfort level with teaching 
the math in this lesson? 



4. How much assistance did you receive from your math 
partner prior to teaching this lesson? 



5. To what degree do you think your students learned the 
math in this lesson? 



6. Overall, how successful was the lesson — both the CTE 
and math components? 



7. Were you able to complete the lesson as planned Yes. 

a. If no, What prevented you from completing it? 

8. Were you able to teach all 7 elements of the math enhancement model? (put X on line) 

Yes. No. If no, what elements were not included? 

9. Do you have any suggestions for improving the teaching of the CTE content or the math 
concepts in this lesson? 



Not at all 



Completely 



Not at all 



Completely 



Low 



High 



None 



A lot 



A little 



A lot 



Not at all 



Completely 



No 
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APPENDIX I. OBSERVATION FORM AND SCORING RUBRIC 



Date:. 

Video: Live:_ 

Number of students in classroom: M F. 



Lesson no.: 

Lesson 

title: 

Note: The observer is expected to study the math-enhanced lesson in advance of the observation. 
Please make general comments about the observation in the space below: 

• Describe barriers, extenuating circumstances, disruptions, unexpected incidents, etc. 

• Describe strengths and successes of the model; what worked. 

• Record anecdotes related to the lesson. 

• Note variations of and additions to the “planned” lesson and instructional materials. 

• If lesson is not completed, note any indications by the teacher that the lesson will be 
completed on another day. 
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Codes for the 7 Elements 

The obserx’er should use the approved math-enhanced lesson plan, with the prescribed activities 
therein, as the guide for assigning codes. However, the obserx’er should not presume a step-by- 
step presentation of the lesson. If a teacher orders the lesson differently, it should be noted as 
such on the observation form. 

1 Teacher introduces the CTE lesson. 

2 Teacher assesses students’ math awareness as it related to the CTE lesson. 

3 Teacher works through the math example embedded in the CTE lesson. 

4 Teacher and students work through related, contextual examples. 

5 Teacher and students work through traditional math examples. 

6 Students demonstrate understanding. 

7 Formal assessment. 



Codes for type of instruction 

These codes will help us learn more about how the enhanced lesson was delivered. These may 
be added by the observer after the lesson is completed. More than one code may be used to 
describe an activity. 



L 


lecture 


CL 


cooperative learning activity 


LD 


lecture with discussion 


LA 


laboratory activity 


Q 


teacher questioning 


WW 


worksheet work/writing 


TD 


teacher demonstration 


T 


use of texts, reading materials 


PM 


teacher problem modeling 


TIS 


teacher interacting w/individual students 


SG 


small group discussion/activity 


A 


assessment of student learning 


SD 


student-led discussion/activity 


R 


review of assignments/tests/projects 


CD 


class discussion 


HW 


assign homework 


HO 


hands-on; experiential activity 


OC 


out-of-classroom 


IN 

UT 


independent student work 

use of computer, calculators, technology 


0 


(field experience, shop, greenhouse, etc.) 
other (please describe) 



Codes for level of student engagement 

HE high engagement (80% of the students engaged) 
ME mixed engagement 

LE low engagement (80% of the students off-task) 
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Record your observations in 5-minute intervals. Note: More than one “element” code may be used in each 
box. 

Min 
0-5 

5-10 

10-15 
15-20 
20-25 
25-30 
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Method (indicate how the lesson was 
taught; note context/location of lesson; 
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collected) 
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Min. 

30-35 



35-40 



40-45 



45-50 



50-55 



55-60 
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Min. 

60-65 



65-70 



70-75 



75-80 



80-85 



85-90 
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Scoring Rubric 

This scoring rubric is a supplement to the observation instrument. It will assist our efforts in 
understanding the extent to which a lesson is implemented. It is to be completed subsequent to 
the classroom observation; scoring must be supported by evidence recorded on the observation 
form. 

A. Check the appropriate boxes in the “complete” and “needs improvement” columns. 

B. Assign each element a score using the following guidelines: 

r 2 3 4 5 

No observable More observable An equal amount of More observable All criteria are 

criteria are met. criteria are unmet observable criteria criteria are met observed. 

than met. are met and unmet. than unmet. 



C. Use comment box to provide justification for the score. 



Elements 


Completed? 


Needs Improvement 


Score 


Comments 


1. Introduce 
the CTE 
lesson. 


_ Specific objectives of CTE 
lesson are explicit on lesson 
plan or in presentation. 

_ Students are introduced to 
the lesson as a CTE lesson. 

_ The math concept embedded 
in the CTE lesson is clearly 
identified — pulled and 
pointed out. 


_ Lesson objectives are 
unclear or not evident. 

_ Little or no introduction 
of the CTE lesson is 
provided to students. 

_ Math concept embedded 
in the CTE lesson is not 
pulled/pointed out or 
made clear to students. 






2. Assess 
students’ 
math 

awareness 
as it relates 
to the CTE 
lesson. 


_ Teacher uses learning 
activities and/or well- 
developed questions to 
assess all students’ 
awareness of the embedded 
math concept. 

_ Teacher uses math 

vocabulary and supporting 
instmctional aids to begin 
bridging of math to CTE. 


_ Teacher does little or 
nothing to assess all 
students’ awareness of the 
embedded math concept. 

_ Teacher uses little or no 
math vocabulary and/or 
instructional aids to 
bridge. 






3. Work 
through the 
math 
example 
embedded 
in the CTE 
lesson. 


_ Teacher uses planned 
steps/processes to work 
through the embedded math 
example. 

_ Teacher explicitly bridges 
CTE and math vocabularies; 
supports with instructional 
strategies and aids. 


_ Steps/processes for 
working through the 
embedded math example 
are incomplete or missing. 

_ Little bridging of CTE 
and math vocabularies is 
evident; few or no strate- 
gies and aids are used to 
relate the CTE to math. 
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4. Work 
through the 
related, 
contextual 
examples. 


_ Teacher provides students 
with opportunities to work- 
through similar examples, 
using the same embedded 
math concept in examples 
from the same occupational 
area. 

_ Example problems are at 
varying levels of difficulty, 
from basic to advanced. 

_ Teacher continues to bridge 
the CTE and math 
vocabularies, supported 
with instructional strategies 
and/or aids. 


_ Few or no additional 
examples of the 
embedded concept are 
provided in the lesson. 

_ Example problems do not 
reflect varying levels of 
difficulty. 

_ Little or no bridging of 
CTE and math 
vocabularies was 
provided or supported 
with instructional 
strategies and/or aids. 






5. Work 
through 
traditional 
math 

examples. 


_ Teacher provides students 
with a variety of examples 
to illustrate the math 
concept as it is presented in 
traditional math tests. 

_ Traditional examples move 
from basic to advanced. 

_ Teacher continues to bridge 
the CTE and math 
vocabularies, supported 
with instructional strategies 
and/or aids. 


_ Few or no math problems 
are provided that illustrate 
the math concept as it is 
presented in standardized 
tests. 

_ Traditional examples do 
not reflect varying levels 
of difficulty. 

_ Little or no bridging of 
CTE and math 
vocabularies is provided 
or supported with 
instructional strategies 
and/or aids. 






6. Students 
demonstrate 
understand- 
ing. 


_ Lesson provides learning 
activities, projects, etc., that 
gives students opportunities 
to demonstrate what they 
have learned. 

_ Lesson ties math examples 
back to the CTE content; 
lesson ends on the CTE 
topic. 


_ No learning activities, 
projects, etc., provide 
students with 
opportunities to 
demonstrate what they 
have learned. 

_ Lesson fails to tie the 
math back to CTE or end 
on the CTE topic. 






7. Formal 
assessment. 


_ Student work is provided to 
show evidence of formal 
assessment in the CTE 
unit/course. 


_ Student work is not 
provided to show 
evidence of formal 
assessments in the CTE 
unit/course. 




Note: Assessments 
will likely occur on 
a day other than that 
observed. Lesson 
plans and examples 
of student work will 
provide evidence of 
this element. 
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APPENDIX J. FOCUS GROUP INTERVIEW MATERIALS 
Focus Group Consent Script 

Thank you for participating in the National Center’s Math-in-CTE project. The purpose of 
this focus group is to learn more about your experiences and will help us plan more effectively 
for Year 2 of the study. 

This focus group discussion will be recorded and transcribed. Notes will also be taken. Every 
effort will be made to ensure confidentiality of the data. The transcriptions will be kept in a 
secure location. 

We also ask that you maintain the confidentiality of each member of our group. To ensure 
confidentiality for others, we also ask that you do not mention names of colleagues, students, 
schools, or other identifiers. 

Your name will be changed to an ID number and will not be associated with any data from 
this interview. In any sort of report we might publish, we will not include any information that 
will make it possible to identify you, your students, or your school. 



Do you have any questions? 

Experimental CTE-Teacher Focus Group Questions 

Oral Consent and Introductions 

1. As you think back over this year, what has it been like to participate in this study? 

How would you describe the process? 

What were the most beneficial aspects of the process? 

2. What was it like to teach the math-enhanced lessons (for a full year) this past year 
for a full year? 

Tell us about it. Probe on: 

• What made it work for you — what didn’t? (barriers) 

• How was it working with a math-teacher partner? 

• How did the experience of participating in this study impact your ability or 
confidence to teach the math-in-CTE? 

3. If you were to advise another school district on how to improve the math skills of their 
CTE students, what would you insist on? What would you recommend but not insist on? 
What do you consider to be the strengths of the model as it has emerged from this study? 
How would you improve or change the model? 
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Probe on the following: 

• Curriculum mapping 

• Professional development 

• Scope and sequence 

• Organizing around specific occupational themes/curricula 

• Meeting with math teachers ahead of lesson 

• Math cluster meetings 

4. How did your students respond to the math-enhanced lessons? To the tests? 

Do you have some stories to share? 

5. What have you learned or gained from the experience of teaching math-in-CTE? 

In what ways, or how, has this experience impacted your teaching? 

To what extent will you continue to teach these lessons? 

Do you plan to enhance the math in other courses? 

6. Are there any final comments? Is there something I haven’t asked that you would like to 
comment on? 

Summary of Notes 

Math-Teacher Focus Group Questions 

Oral Consent and Introductions 

1. As you think back over this year, what has it been like to participate in this study? 

How would you describe the process? 

What were the most beneficial aspects of the process? 

2. From your perspective in providing support throughout the year, what can you tell us 
about the math-enhanced lessons? 

• What made it work for your CTE partner? What didn’t? [barriers] 

• How was it working with a CTE teacher-partner? 

• How did the experience impact your understanding/perspective on CTE? 

3. If you were to advise another school district on how to improve the math skills of their 
CTE students, 

What would you insist on? What would you recommend but not insist on? 

What do you consider to be the strengths of the model as it has emerged? 

How would you improve or change the model? 
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Probe on the following: 

• Curriculum mapping 

• Professional development 

• Scope and sequence 

• Organizing around specific occupational themes/curricula 

• Math meetings with the CTE teacher 

• Math clusters (if they were involved) 

4. How did students respond to the math-enhanced lessons? To the tests? 

Do you have some stories to share? 

5. What have you learned or gained from the experience of participating in the study? 

In what ways, or how, has this experience impacted your teaching? 

6. Are there any final comments? 

Is there something I haven’t asked that you would like to comment on? 

Summary of Notes 

Mixed Focus Group Questions 

Oral Consent and Introductions 

1. As you think back over this year, what has it been like to participate in this study? 

How would you describe the process? 

What were the most beneficial aspects of the process? 

2. What can you tell us about the math-enhanced lessons (for a full year)? Probe on: 

• What made it work — what didn’t? (barriers) 

• How was it working as teacher-partners? 

• CTE: How did the experience of participating in this study impact your ability or 
confidence to teach the math-in-CTE? 

• Math: How did the experience impact your perspectives on math-in-CTE? 

3. If you were to advise another school district on how to improve the math skills of their 
CTE students, what would you insist on? What would you recommend but not insist on? 
What do you consider to be the strengths of the model as it has emerged from this study? 
How would you improve or change the model? Probe on the following: 

• Curriculum mapping 

• Professional development 

• Scope and sequence 

• Organizing around specific occupational themes/curricula 

• Meeting together ahead of each lesson 

• Math cluster meetings 
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4. How did students respond to the math-enhanced lessons? To the tests? 

Do you have some stories to share? 

5. What have you learned or gained from the experience of teaching math-in-CTE? 

In what ways, or how, has this experience impacted your teaching? 

( CTE only, if appropriate): To what extent will you continue to teach these lessons? 

Do you plan to enhance the math in other courses? 

6. Are there any final comments? Is there something I haven’t asked that you would like to 
comment on? 

Summary of Notes 
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APPENDIX K. PRETEST RESULTS 

The first step in the quantitative analysis was to determine whether the random assignment at 
the classroom level had yielded experimental and control groups that were comparable on the 
pretest. The results in Appendix Table 1 for “All sites” indicate that prior to the experimental 
treatment the mean scores of the two groups were equivalent. However, comparisons for each 
occupational area revealed that there were pretest differences in Sites A and F. Therefore, pretest 
scores were used as a control variable in all subsequent analyses. 

Appendix Table 1 

Mean Classroom Pretest Scores of Experimental and Control Groups by SLMP 



Experimental Control 



SLMP Site 


M 


SD 


n 


M 


SD 


N 


t 


P 


A 


68.86 


0.84 


2 


54.83 


6.90 


4 


2.703 


0.054 


B 


43.08 


5.77 


11 


47.06 


6.82 


17 


-1.597 


0.122 


C 


45.93 


9.59 


21 


45.39 


12.09 


23 


0.164 


0.870 


E 


54.20 


8.88 


9 


55.40 


9.47 


14 


-0.306 


0.763 


F 


42.88 


7.54 


14 


49.80 


8.06 


16 


-2.416 


0.022 


All 


46.74 


9.81 


57 


49.13 


10.02 


74 


-1.365 


0.175 



Note. Probabilities are for two-tailed tests with equal variances assumed. 



National Research Center for Career and Technical Education 



149 




Building Academic Skills in Context: Testing the Value of Enhanced Math Learning in CTE 



APPENDIX L. STUDY DESIGN 



NRCCTE 

Researchers, 

Professional Development Trainers 



Site Facilitator/ 
Testing Liaisons/ 
Math Captains 1 



SLMPB 

Site Facilitator/ 
Testing Liaisons/ 
Math Captains 



SLMPC 

Site Facilitator/ 
Testing Liaisons/ 
Math Captains 



SLMPE 

Site Facilitator/ 
Testing Liaisons/ 
Math Captains 



Site Facilitator/ 
Testing Liaisons/ 
Math Captains 




'As explained in the report, math captains were chosen from the group of math-teacher 
partners for that SLMP to act as coaches for the experimental CTE teachers at regional 
cluster meetings in between each of the professional development workshops. 
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